One-Step Distillation and the Limits of Fast Generation

Reliable deployment in compressed models serving multilingual outputs depends on more than obtaining a strong benchmark result. This conceptual analysis uses efficient generation to study how claims travel from data to model output and then to action. Its central thesis is that the unit of assurance must be the trajectory from tokenization and draft generation through scoring, revision, compression, and release. The reviewed evidence shows recurring risks from hidden distribution change, correlated evaluation error, missing provenance, and optimization objectives that omit downstream costs. In response, the article proposes a layered evaluation program combining controlled perturbations, subgroup and scenario analysis, repeated runs, calibration or selective prediction, and monitoring after release. It also asks who can inspect, override, and learn from failures. By integrating the assigned target papers with established scholarship, the synthesis clarifies which findings transfer across domains and which remain local to a benchmark, dataset, or experimental apparatus. The goal is a testable research program for bounded, traceable, and revisable systems.

1. Zhang, Yin, et al. "SAINF: Intrinsic Self-Correction for Robust Machine Translation with Large Language Models." *Frontiers of Computer Science* (2026).
2. Li, Yuanhao, et al. "DRAFT-RL: Multi-Agent Chain-of-Draft Reasoning for Reinforcement Learning-Enhanced LLMs." *Proceedings of the AAAI Conference on Artificial Intelligence* 40.35 (2026): 29530-29537.
3. Chen, Yiwei, et al. "One-Step Generative Distillation." *ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)* (2026).
4. Rafailov, Rafael, et al. "Direct Preference Optimization: Your Language Model Is Secretly a Reward Model." *Advances in Neural Information Processing Systems*, vol. 36, 2023.
5. Sutton, Richard S., and Andrew G. Barto. *Reinforcement Learning: An Introduction*. 2nd ed., MIT Press, 2018.
6. Tishby, Naftali, Fernando C. Pereira, and William Bialek. "The Information Bottleneck Method." *Proceedings of the 37th Annual Allerton Conference on Communication, Control, and Computing*, 1999, pp. 368-377.
7. Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. "Distilling the Knowledge in a Neural Network." *NIPS Deep Learning and Representation Learning Workshop*, 2015.
8. Guo, Chuan, et al. "On Calibration of Modern Neural Networks." *Proceedings of the 34th International Conference on Machine Learning*, 2017, pp. 1321-1330.
9. Geifman, Yonatan, and Ran El-Yaniv. "Selective Classification for Deep Neural Networks." *Advances in Neural Information Processing Systems*, vol. 30, 2017.
10. Araci, Dogu. "FinBERT: Financial Sentiment Analysis with Pre-Trained Language Models." *arXiv preprint arXiv:1908.10063*, 2019.
11. Loughran, Tim, and Bill McDonald. "When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10-Ks." *Journal of Finance*, vol. 66, no. 1, 2011, pp. 35-65.
12. Gururangan, Suchin, et al. "Don't Stop Pretraining: Adapt Language Models to Domains and Tasks." *Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics*, 2020, pp. 8342-8360.
13. Vaswani, Ashish, et al. "Attention Is All You Need." *Advances in Neural Information Processing Systems*, vol. 30, 2017.