Less Is More: Rethinking Parameter-Efficient Fine-Tuning from a Subtractive Perspective

Tianqi Jiang; Liu Yang; Xi-Le Zhao; Zixuan Qin; Qinghua Hu

AAAI 2026technical0 citations

Less Is More: Rethinking Parameter-Efficient Fine-Tuning from a Subtractive Perspective

Tianqi Jiang, Liu Yang, Xi-Le Zhao, Zixuan Qin, Qinghua Hu

Abstract

Currently, pretrained models are rapidly scaling in size, which substantially increases the cost of fine-tuning them for downstream tasks. To address this challenge, parameter-efficient fine-tuning (PEFT) methods have been developed to optimize a minimal set of parameters for adaptation. While current PEFT approaches predominantly employ an "additive

BibTeX

@inproceedings{aaai2026_lessismorerethin,
  title = {Less Is More: Rethinking Parameter-Efficient Fine-Tuning from a Subtractive Perspective},
  author = {Tianqi Jiang and Liu Yang and Xi-Le Zhao and Zixuan Qin and Qinghua Hu},
  booktitle = {AAAI 2026},
  year = {2026}
}