ASPO: Adaptive Sentence-Level Preference Optimization for Fine-Grained Multimodal Reasoning
Direct Preference Optimization (DPO) has gained significant attention for its simplicity and computational efficiency in aligning large language models (LLMs). Recent advancements have extended DPO to multimodal scenarios, achieving strong performance. However, traditional DPO relies on binary prefe…