FeifeiTong

Eliminating Inductive Bias in Reward Models with Information-Theoretic Guidance

Reward models (RMs) are crucial in reinforcement learning from human feedback (RLHF) to align large language models (LLMs) with human values. However, RM training data is commonly recognized as low-quality, always containing preference conflicts and inductive biases, such as response length or speak…