SWIFT: A Scalable Lightweight Infrastructure for Fine-Tuning
Yuze Zhao, Jintao Huang, Jinghan Hu, Xingjun Wang, Yunlin Mao, Daoze Zhang, Zeyinzi Jiang, Zhikai Wu
Abstract
Recent development in Large Language Models (LLMs) and Multi-modal Large Language Models (MLLMs) have achieved superior performance and generalization capabilities, covered extensive areas of traditional tasks. However, existing large model training frameworks support only a limited number of models and techniques, particularly lacking in support for new models, which makes fine-tuning LLMs challenging for most developers. Therefore, we develop SWIFT, a customizable one-stop infrastructure for large models. With support of over 350+ LLMs and 80+ MLLMs, SWIFT stands as the open-source framework that provide the most comprehensive support for fine-tuning large models. In particular, it is the first training framework that provides systematic support for MLLMs. Moreover, SWIFT integrates post-training processes such as inference, evaluation, and quantization, to facilitate fast adoptions of large models in various application scenarios, offering helpful utilities like benchmark comparisons among different training techniques.
BibTeX
@article{Zhao_Huang_Hu_Wang_Mao_Zhang_Jiang_Wu_Ai_Wang_Zhou_Chen_2025, title={SWIFT: A Scalable Lightweight Infrastructure for Fine-Tuning}, volume={39}, url={https://ojs.aaai.org/index.php/AAAI/article/view/35383}, DOI={10.1609/aaai.v39i28.35383}, abstractNote={Recent development in Large Language Models (LLMs) and Multi-modal Large Language Models (MLLMs) have achieved superior performance and generalization capabilities, covered extensive areas of traditional tasks. However, existing large model training frameworks support only a limited number of models and techniques, particularly lacking in support for new models, which makes fine-tuning LLMs challenging for most developers. Therefore, we develop SWIFT, a customizable one-stop infrastructure for large models. With support of over 350+ LLMs and 80+ MLLMs, SWIFT stands as the open-source framework that provide the most comprehensive support for fine-tuning large models. In particular, it is the first training framework that provides systematic support for MLLMs. Moreover, SWIFT integrates post-training processes such as inference, evaluation, and quantization, to facilitate fast adoptions of large models in various application scenarios, offering helpful utilities like benchmark comparisons among different training techniques.}, number={28}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Zhao, Yuze and Huang, Jintao and Hu, Jinghan and Wang, Xingjun and Mao, Yunlin and Zhang, Daoze and Jiang, Zeyinzi and Wu, Zhikai and Ai, Baole and Wang, Ang and Zhou, Wenmeng and Chen, Yingda}, year={2025}, month={Apr.}, pages={29733-29735} }