Adapters Selector: Cross-domains and Multi-tasks LoRA Modules Integration Usage Method
Yimin Tian, Bolin Zhang, Zhiying Tu, Dianhui Chu
Abstract
Parameter-Efficient Fine-Tuning (PEFT) adapts large language models (LLMs) to specific domains by updating only a small portion of the parameters. Although fine-tuning on a single task within a specific domain has demonstrated promising results, there remains limited exploration on how to effectively integrate these adapters for optimal performance. In this paper, we propose Adapters Selector (AS): a novel framework for better integrating usage of multiple adapters by training a middleman adapter to select the appropriate adapter for inference. Our approach utilizes PEFT to train a selector that determines which input content corresponds to which task in which domain, and subsequently selects the homologous adapter. By the way, The AS has developed the capability to execute cross-domain multi-tasks effectively through the utilization of a compact model in combination with multiple LoRA modules. Our code is publicly available.
BibTeX
@inproceedings{tian-etal-2025-adapters,
title = "Adapters Selector: Cross-domains and Multi-tasks {L}o{RA} Modules Integration Usage Method",
author = "Tian, Yimin and
Zhang, Bolin and
Tu, Zhiying and
Chu, Dianhui",
editor = "Rambow, Owen and
Wanner, Leo and
Apidianaki, Marianna and
Al-Khalifa, Hend and
Eugenio, Barbara Di and
Schockaert, Steven",
booktitle = "Proceedings of the 31st International Conference on Computational Linguistics",
month = jan,
year = "2025",
address = "Abu Dhabi, UAE",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.coling-main.40/",
pages = "593--605"
}