2026
Position: Measuring Human Preferences in RLHF is a Social Science Problem
ICML 2026spotlight
RLHF assumes that annotation responses reflect genuine human preferences. We argue this assumption warrants systematic examination, and that behavioral science offers frameworks that bring clarity to when it holds and when it breaks down. Behavioral scientists have documented for sixty years that pe…