2026
Beyond Binary Preferences: A Principled Framework for Reward Modeling with Ordinal Feedback
ICLR 2026poster
Reward modeling is crucial for aligning large language models with human preferences, yet current approaches lack a principled mathematical framework for leveraging ordinal preference data. When human annotators provide graded preferences on a Likert scale (e.g., significantly better, better, slight…