Large Language Model Unlearning for Source Code
Xue Jiang, Yihong Dong, Huangzhao Zhang, Tangxinyu Wang, Zheng Fang, Yingwei Ma, Rongyu Cao, Binhua Li
Abstract
While Large Language Models (LLMs) excel at code generation, their inherent tendency toward verbatim memorization of training data introduces critical risks like copyright infringement, insecurity emission, and deprecated API utilization, etc. A straightforward yet promising defense is unlearning, i.e., erasing or down-weighting the offending snippets through post-training. However, we find its application to source code often tends to spill over, damaging the basic knowledge of programming languages learned by the LLM and degrading the overall capability. To ease this challenge, we propose PROD for precise source code unlearning. PROD surgically zeroes out the prediction probability of the prohibited tokens, and renormalizes the remaining distribution so that the generated code stays correct. By excising only the targeted snippets, PROD achieves precise forgetting without much degradation of the LLM
BibTeX
@inproceedings{aaai2026_largelanguagemod,
title = {Large Language Model Unlearning for Source Code},
author = {Xue Jiang and Yihong Dong and Huangzhao Zhang and Tangxinyu Wang and Zheng Fang and Yingwei Ma and Rongyu Cao and Binhua Li and Zhi Jin and Wenpin Jiao and Yongbin Li and Ge Li},
booktitle = {AAAI 2026},
year = {2026}
}