Ambiguity Awareness Optimization: Towards Semantic Disambiguation for Direct Preference Optimization
Direct Preference Optimization (DPO) is a widely used reinforcement learning from human feedback (RLHF) method across various domains. The study of token importance has attracted widespread attention in DPO. Researchers have found that token importance is crucial for improving the effectiveness of D