2026
Beyond Temperature: Hyperfitting as a Late-Stage Geometric Expansion
ICML 2026poster
Recent work has identified a counterintuitive phenomenon termed “Hyperfitting", where fine-tuning Large Language Models (LLMs) to near-zero training loss on small datasets surprisingly enhances open-ended generation quality and mitigates repetition in greedy decoding. While effective, the underlying…