AetherCode: Evaluating LLMs’ Ability to Win In Premier Programming Competitions
Zihan Wang, Jiaze Chen, Zhicheng Liu, Haojie Pan, Markus Mak, Yidi Du, Geonsik Moon, Aaron Tua
Abstract
Competitive programming has emerged as a critical benchmark for evaluating the reasoning and coding capabilities of Large Language Models (LLMs). Despite impressive progress on existing benchmarks, we argue that current evaluations overstate model proficiency, masking a substantial gap between LLMs and elite human programmers. This gap arises from two key limitations: insufficient difficulty and scope of benchmark problems, and evaluation bias from low-quality test cases. To address these shortcomings, we present **AetherCode**, a new benchmark that draws problems from premier programming competitions such as IOI and ICPC, offering broader coverage and higher difficulty. AetherCode further incorporates comprehensive, expert-validated test suites built through a hybrid of automated generation and human curation, ensuring rigorous and reliable assessment. By combining challenging problem design with robust evaluation, AetherCode provides a more faithful measure of LLM capabilities and sets a new standard for future research in code reasoning.
BibTeX
@inproceedings{
wang2026aethercode,
title={AetherCode: Evaluating {LLM}s{\textquoteright} Ability to Win In Premier Programming Competitions},
author={Zihan Wang and Jiaze Chen and Zhicheng Liu and Haojie Pan and Markus Mak and Yidi Du and Geonsik Moon and Aaron Tua and Kunshuo Peng and Jiayi Lu and Boqian Zou and Chenyang Ran and GuangTian and Shoutai Zhu and Duan Yeheng and Zhenghui Kang and Zhenxing Lin and Lishangshu and Qiang Luo and Qingshen Long and Zhiyong Chen and Yihan Xiao and Yurong Wu and Daoguang Zan and Mingxuan Wang and Ming Ding},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026},
url={https://openreview.net/forum?id=lSM6MtjQcM}
}