2026
The Power of Power Law: Asymmetry Enables Compositional Reasoning
ICML 2026spotlight
Natural language data follows a power-law distribution, with most knowledge and skills appearing at very low frequency. While a common intuition suggests that reweighting or curating data toward a uniform distribution may help models better learn these long-tail skills, we find a counterintuitive re…