John D Patterson

Diverse, not Short: A Length-Controlled Data Selection Strategy for Improving Response Diversity of Language Models

Diverse language model responses are crucial for creative generation, open-ended tasks, and self-improvement training. We show that common diversity metrics, and even reward models used for preference optimization, systematically bias models toward shorter outputs, limiting expressiveness. To addres