2026
Position: Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility
ICML 2026poster
Code-related benchmarks play a critical role in evaluating large language models (LLMs), yet their quality fundamentally shapes how the com- munity interprets model capabilities. In the past few years, awareness of benchmark quality has grown. Yet, after a decade-scale (2014 - 2025) survey over 572 …