DualOptim+: Bridging Shared and Decoupled Optimizer States for Better Machine Unlearning in Large Language Models
Xuyang Zhong, Qizhang Li, Yiwen Guo, Chen Liu
Abstract
We propose **DualOptim+**, a novel optimization framework for improving machine unlearning in large language models. It introduces a base state to capture common representations shared by forgetting and retaining objectives and delta states to preserve objective-specific residuals. This architecture allows the optimizer to adaptively bridge shared and decoupled states based on the directional conflict between forgetting and retaining gradients. We further introduce DualOptim+ 8bit, a quantized variant that reduces memory overhead without compromising performance. Extensive experiments across fictitious, real-world, and safety alignment tasks demonstrate that DualOptim+ consistently achieves a superior trade-off between forgetting efficacy and model utility.
BibTeX
@inproceedings{
zhong2026dualoptim,
title={DualOptim+: Bridging Shared and Decoupled Optimizer States for Better Machine Unlearning in Large Language Models},
author={Xuyang Zhong and Qizhang Li and Steven Y. Guo and Chen Liu},
booktitle={Forty-third International Conference on Machine Learning},
year={2026},
url={https://openreview.net/forum?id=xGfnQajGPQ}
}