Parameter-Efficient Fine-Tuning of Large Language Models via Deconvolution in Subspace

Jia-Chen Zhang; Yu-Jie Xiong; Chun-Ming Xia; Dong-Hai Zhu; Xi-He Qiu

COLING 2025main2 citations

Parameter-Efficient Fine-Tuning of Large Language Models via Deconvolution in Subspace

Jia-Chen Zhang, Yu-Jie Xiong, Chun-Ming Xia, Dong-Hai Zhu, Xi-He Qiu

Abstract

This paper proposes a novel parameter-efficient fine-tuning method that combines the knowledge completion capability of deconvolution with the subspace learning ability, reducing the number of parameters required for fine-tuning by 8 times . Experimental results demonstrate that our method achieves superior training efficiency and performance compared to existing models.

BibTeX

@inproceedings{zhang-etal-2025-parameter,
    title = "Parameter-Efficient Fine-Tuning of Large Language Models via Deconvolution in Subspace",
    author = "Zhang, Jia-Chen  and
      Xiong, Yu-Jie  and
      Xia, Chun-Ming  and
      Zhu, Dong-Hai  and
      Qiu, Xi-He",
    editor = "Rambow, Owen  and
      Wanner, Leo  and
      Apidianaki, Marianna  and
      Al-Khalifa, Hend  and
      Eugenio, Barbara Di  and
      Schockaert, Steven",
    booktitle = "Proceedings of the 31st International Conference on Computational Linguistics",
    month = jan,
    year = "2025",
    address = "Abu Dhabi, UAE",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.coling-main.265/",
    pages = "3924--3935"
}