ICLR 2026poster0 citations
High-Probability Bounds for the Last Iterate of Clipped SGD
Savelii Chezhegov, Daniela Angela Parletta, Andrea Paudice, Eduard Gorbunov
Abstract
We study the problem of minimizing a convex objective when only noisy gradient estimates are available. Under the mild assumption that the stochastic gradients have finite $\alpha$-th moments for some $\alpha \in (1,2]$, we show that the last iterate of clipped stochastic gradient descent (Clipped-SGD) achieves high-probability convergence of order $1/K^{(2\alpha-2)/(3\alpha)}$ on smooth objectives. Finally, we provide empirical results that support and complement our theoretical analysis.
Stochastic Optimization
BibTeX
@inproceedings{
chezhegov2026highprobability,
title={High-Probability Bounds for the Last Iterate of Clipped {SGD}},
author={Savelii Chezhegov and Daniela Angela Parletta and Andrea Paudice and Eduard Gorbunov},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026},
url={https://openreview.net/forum?id=4sGEvpwyxN}
}