TuckA: Hierarchical Compact Tensor Experts for Efficient Fine-Tuning

Qifeng Lei; Zhiyong Yang; Qianqian Xu; Cong Hua; Peisong Wen; Qingming Huang

AAAI 2026technical0 citations

TuckA: Hierarchical Compact Tensor Experts for Efficient Fine-Tuning

Qifeng Lei, Zhiyong Yang, Qianqian Xu, Cong Hua, Peisong Wen, Qingming Huang

Abstract

Efficiently fine-tuning pre-trained models for downstream tasks is a key challenge in the era of foundation models. Parameter-efficient fine-tuning (PEFT) presents a promising solution, achieving performance comparable to full fine-tuning by updating only a small number of adaptation weights per layer. Traditional PEFT methods typically rely on a single expert, where the adaptation weight is a low-rank matrix. However, for complex tasks, the data

BibTeX

@inproceedings{aaai2026_tuckahierarchica,
  title = {TuckA: Hierarchical Compact Tensor Experts for Efficient Fine-Tuning},
  author = {Qifeng Lei and Zhiyong Yang and Qianqian Xu and Cong Hua and Peisong Wen and Qingming Huang},
  booktitle = {AAAI 2026},
  year = {2026}
}