2026
Position: LLM Benchmark Datasets should be Contamination-Resistant
ICML 2026poster
Benchmark datasets are critical for reproducible, reliable and discriminative evaluation of LLMs. However, recent studies reveal that many benchmark datasets are included in pretraining corpora, i.e. *contaminated*, which diminishes their value as a reliable measure of model generalization. In this …