Bijean Ghafouri

Position: Measuring Human Preferences in RLHF is a Social Science Problem

RLHF assumes that annotation responses reflect genuine human preferences. We argue this assumption warrants systematic examination, and that behavioral science offers frameworks that bring clarity to when it holds and when it breaks down. Behavioral scientists have documented for sixty years that pe…