Niladri S. Chatterji — accepted AI-conference papers

2025

Compute Optimal Scaling of Skills: Knowledge vs Reasoning

Nicholas Roberts, Niladri S. Chatterji, Sharan Narang, Mike Lewis, Dieuwke Hupkes

ACL 2025finding

Scaling laws are a critical component of the LLM development pipeline, most famously as a way to forecast training decisions such as ‘compute-optimally’ trading-off parameter count and dataset size, alongside a more recent growing list of other crucial decisions. In this work, we ask whether compute…

Cited by 0 Source ↗PDF ↗Scholar ↗

2024

Proving Test Set Contamination in Black-Box Language Models

Yonatan Oren, Nicole Meister, Niladri S. Chatterji, Faisal Ladhak, Tatsunori Hashimoto

ICLR 2024oral

Large language models are trained on vast amounts of internet data, prompting concerns that they have memorized public benchmarks. Detecting this type of contamination is challenging because the pretraining data used by proprietary models are often not publicly accessible. We propose a procedure fo…

Cited by 137 Source ↗PDF ↗code ↗Scholar ↗