CJ Y

When Labelers Stay Silent: The Power of Ties in Cost-Effective Preference Learning

Standard preference alignment relies on a binary forced-choice paradigm, assuming definitive preferences for all pairs. However, we find that indistinguishable pairs are prevalent even in standard benchmarks, where quality differences of two responses often fall below the labeler's discriminative re…