Daniel A. Keim

ResponseRank: Data-Efficient Reward Modeling through Preference Strength Learning

Binary choices, as often used for reinforcement learning from human feedback (RLHF), convey only the *direction* of a preference. A person may choose apples over oranges and bananas over grapes, but *which preference is stronger*? Strength is crucial for decision-making under uncertainty and general…