Jodi Casabianca

Noise Tectonics: Measuring the Stability of AI Benchmark Ecosystems

AI benchmark ecosystems compress rich evaluation data into aggregate leaderboard scores, but these scores contain substantial measurement noise whose sources and magnitudes remain unquantified. Without systematic methods to measure this noise and separate signal from artifact, it is unclear when ben…