Benjamin Klieger

ResearchCodeBench: Benchmarking LLMs on Implementing Novel Machine Learning Research Code

Large language models (LLMs) have shown promise in transforming machine learning research, yet their capability to faithfully implement genuinely novel ideas from recent research papers—ideas unseen during pretraining—remains unclear. We introduce ResearchCodeBench, a benchmark that evaluates LLMs’…