2026
From Optimization to Generalization under Heavy-Tailed Data: The Role of Gradient Clipping
ICML 2026poster
Gradient clipping is widely used to stabilize stochastic gradient methods and is often theoretically motivated by heavy-tailed gradient noise, where even second moments may be infinite, seemingly contradicting empirical risk minimization where all moments are finite for a fixed dataset. We resolve t…