C2LEVA: Toward Comprehensive and Contamination-Free Language Model Evaluation
Yanyang Li, Wong Tin Long, Cheung To Hung, Jianqiao Zhao, Duo Zheng, Liu Ka Wai, Michael R. Lyu, Liwei Wang
Abstract
Recent advances in large language models (LLMs) have shown significant promise, yet their evaluation raises concerns, particularly regarding data contamination due to the lack of access to proprietary training data. To address this issue, we present C2LEVA, a comprehensive bilingual benchmark featuring systematic contamination prevention. C2LEVA firstly offers a holistic evaluation encompassing 22 tasks, each targeting a specific application or ability of LLMs, and secondly a trustworthy assessment due to our contamination-free tasks, ensured by a systematic contamination prevention strategy that fully automates test data renewal and enforces data protection during benchmark data release. Our large-scale evaluation of 15 open-source and proprietary models demonstrates the effectiveness of C2LEVA.
BibTeX
@inproceedings{li-etal-2025-c2leva,
title = "{C}$^2${LEVA}: Toward Comprehensive and Contamination-Free Language Model Evaluation",
author = "Li, Yanyang and
Long, Wong Tin and
Hung, Cheung To and
Zhao, Jianqiao and
Zheng, Duo and
Wai, Liu Ka and
Lyu, Michael R. and
Wang, Liwei",
editor = "Che, Wanxiang and
Nabende, Joyce and
Shutova, Ekaterina and
Pilehvar, Mohammad Taher",
booktitle = "Findings of the Association for Computational Linguistics: ACL 2025",
month = jul,
year = "2025",
address = "Vienna, Austria",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.findings-acl.116/",
doi = "10.18653/v1/2025.findings-acl.116",
pages = "2283--2306",
ISBN = "979-8-89176-256-5"
}