Tianxi Zhu

Stochastic Gradient Methods under Heavy-Tailed Noises in Weakly Convex Optimization

Recently, many empirical work has shown that, in machine learning, the noise distribution of stochastic gradients often exhibits heavy tails when stochastic optimization methods are employed. Most existing theoretical analyses of heavy-tailed stochastic methods rely on various convexity and smoothne…