AAAI 2026technical0 citations
Bias-Restrained Prefix Representation Finetuning for Mathematical Reasoning
Sirui Liang, Pengfei Cao, Jian Zhao, Cong Huang, Jun Zhao, Kang Liu
Abstract
Parameter-Efficient finetuning (PEFT) enhances model performance on downstream tasks by updating a minimal subset of parameters. Representation finetuning (ReFT) methods further improve efficiency by freezing model weights and optimizing internal representations with fewer parameters than PEFT, outperforming PEFT on several tasks. However, ReFT exhibits a significant performance decline on mathematical reasoning tasks. To address this problem, the paper demonstrates that ReFT
BibTeX
@inproceedings{aaai2026_biasrestrainedpr,
title = {Bias-Restrained Prefix Representation Finetuning for Mathematical Reasoning},
author = {Sirui Liang and Pengfei Cao and Jian Zhao and Cong Huang and Jun Zhao and Kang Liu},
booktitle = {AAAI 2026},
year = {2026}
}