Autoregressive, Yet Revisable: In Decoding Revision for Secure Code Generation
Chengran Yang, zichao wei, Heminghao Deng, Jinfeng Jiang, Zhensu Sun, Ting Zhang, Tianyi Wu, Ming Wen
Abstract
Large Language Model (LLM) based code generation is predominantly formulated as a strictly monotonic process, appending tokens linearly to an immutable prefix. This formulation contrasts to the cognitive process of programming, which is inherently interleaved with forward generation and on-the-fly revision. While prior works attempt to introduce revision via post-hoc agents or external static tools, they either suffer from high latency or fail to leverage the model's intrinsic semantic reasoning. In this paper, we propose Stream of Revision, a paradigm shift that elevates code generation from a monotonic stream to a dynamic, self-correcting trajectory by leveraging model's intrinsic capabilities. We introduce specific action tokens that enable the model to seamlessly backtrack and edit its own history within a single forward pass. By internalizing the revision loop, our framework Stream of Revision allows the model to activate its latent capabilities just-in-time without external dependencies. Empirical results on secure code generation show that Stream of Revision significantly reduces vulnerabilities with minimal inference overhead. We release our code and dataset at \url{https://anonymous.4open.science/r/Stream-of-Revision-CB74/}, and models will be released upon publication.
BibTeX
@inproceedings{
yang2026autoregressive,
title={Autoregressive, Yet Revisable: In Decoding Revision for Secure Code Generation},
author={Chengran Yang and zichao wei and Heminghao Deng and Jinfeng Jiang and Zhensu Sun and Ting Zhang and Tianyi Wu and Ming Wen and David Lo},
booktitle={Forty-third International Conference on Machine Learning},
year={2026},
url={https://openreview.net/forum?id=QrOWZ2GkI0}
}