Ruibo Deng

AMaPO: Adaptive Margin-attached Preference Optimization for Language Model Alignment

Offline preference optimization offers a simpler and more stable alternative to RLHF for aligning language models. However, their effectiveness is critically dependent on ranking accuracy, a metric where further gains are highly impactful. This limitation arises from a fundamental problem that we id