2024
Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?
NeurIPS 2024poster
Performance on popular ML benchmarks is highly correlated with model scale, suggesting that most benchmarks tend to measure a similar underlying factor of general model capabilities. However, substantial research effort remains devoted to designing new benchmarks, many of which claim to measure nove…