COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning
Yuelin Bai, Xeron Du, Yiming Liang, Leo Jin, Junting Zhou, Ziqiang Liu, Feiteng Fang, Mingshan Chang
Abstract
Remarkable progress on large language models (LLMs), particularly in English, has facilitated impressive capabilities in following human instructions. However, there remains a noticeable gap in instruction fine-tuning for Chinese, where the complex linguistic features pose significant challenges. Existing datasets, generally distilled from English-centric LLMs, are not well-aligned with Chinese users’ interaction patterns. To bridge this gap, we introduce COIG-CQIA, a new Chinese instruction tuning dataset derived from various real-world data resources and undergoing comprehensive human verification. We conduct extensive experiments on COIG-CQIA, and compare them with strong baseline models and datasets. The experimental results show that models trained on COIG-CQIA achieve highly competitive performance in diverse benchmarks. Additionally, our findings offer several insights for designing effective Chinese instruction-tuning datasets and data mixing strategies. Our dataset are available at https://huggingface.co/datasets/m-a-p/COIG-CQIA.
BibTeX
@inproceedings{bai-etal-2025-coig,
title = "{COIG}-{CQIA}: Quality is All You Need for {C}hinese Instruction Fine-tuning",
author = "Bai, Yuelin and
Du, Xeron and
Liang, Yiming and
Jin, Leo and
Zhou, Junting and
Liu, Ziqiang and
Fang, Feiteng and
Chang, Mingshan and
Zheng, Tianyu and
Zhang, Xincheng and
Ma, Nuo and
Wang, Zekun Moore and
Yuan, Ruibin and
Wu, Haihong and
Lin, Hongquan and
Huang, Wenhao and
Zhang, Jiajun and
Lin, Chenghua and
Fu, Jie and
Yang, Min and
Ni, Shiwen and
Zhang, Ge",
editor = "Chiruzzo, Luis and
Ritter, Alan and
Wang, Lu",
booktitle = "Findings of the Association for Computational Linguistics: NAACL 2025",
month = apr,
year = "2025",
address = "Albuquerque, New Mexico",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.findings-naacl.457/",
pages = "8190--8205",
ISBN = "979-8-89176-195-7"
}