RECODE: A Benchmark for Research Code DEvelopment with Interactive Human Feedback
Chunyu Miao, Henry Peng Zou, Yangning Li, Yankai Chen, Yibo Wang, Fangxin Wang, Yifan Li, Wooseong Yang
Abstract
Large language models (LLMs) show the promise in supporting scientific research implementation, yet their ability to generate correct and executable code remains limited. Existing works largely adopt one-shot settings, ignoring the iterative and feedback-driven nature of realistic workflows of scientific research development. To address this gap, we present RECODE, a benchmark of 102 tasks from research papers and repositories that evaluates LLMs through multi-turn interactions with human feedback. It includes structured instructions, unit tests, and a five-level feedback hierarchy to reflect realistic researcher–agent collaboration. We further present ReCodeAgent, a framework that integrates feedback into iterative code generation. Experimentswith leading LLMs, including GPT-5, Claude-Sonnet-4, DeepSeek-V3.1, and Gemini 2.5, show substantial performance gains with richer feedback, while also highlighting ongoing challenges in the generation of complex research code. RECODE establishes a foundation for developing adaptive, feedback-driven LLM agents in scientific research implementation.
BibTeX
@inproceedings{
miao2026recode,
title={{RECODE}: A Benchmark for Research Code {DE}velopment with Interactive Human Feedback},
author={Chunyu Miao and Henry Peng Zou and Yangning Li and Yankai Chen and Yibo Wang and Fangxin Wang and Yifan Li and Wooseong Yang and Bowei He and Xinni Zhang and Dianzhi Yu and Hanchen Yang and Hoang H Nguyen and Yue Zhou and Jie Yang and Jizhou Guo and Wenzhe Fan and Chin-Yuan Yeh and Panpan Meng and Liancheng Fang and Jinhu Qi and Wei-Chieh Huang and Zhengyao Gu and Yuwei Han and Langzhou He and Yuyao Yang and Xue Liu and Irwin King and Philip S. Yu},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026},
url={https://openreview.net/forum?id=IKnuyyPHCV}
}