Ma yinuo

The Quality-Utility Paradox: Why High-Reward Data Impairs Small Model Reasoning

Knowledge distillation from powerful reasoning models underpins the development of Small Language Models (SLMs). A prevailing assumption in this paradigm is that training data with higher perceived quality, often defined by rigorous logic and superior reward scores, monotonically enhances downstream…