NeurIPS 2025poster0 citations
Convergence of Clipped SGD on Convex $(L_0,L_1)$-Smooth Functions
Ofir Gaash, Kfir Yehuda Levy, Yair Carmon
Abstract
We study stochastic gradient descent (SGD) with gradient clipping on convex functions under a generalized smoothness assumption called $(L_0,L_1)$-smoothness. Using gradient clipping, we establish a high probability convergence rate that matches the SGD rate in the $L$ smooth case up to polylogarithmic factors and additive terms. We also propose a variation of adaptive SGD with gradient clipping, which achieves the same guarantee. We perform empirical experiments to examine our theory and algorithmic choices.
convex optimizationstochastic optimizationsmooth optimizationgeneralized smoothness
BibTeX
@inproceedings{
gaash2025convergence,
title={Convergence of Clipped {SGD} on Convex \$(L\_0,L\_1)\$-Smooth Functions},
author={Ofir Gaash and Kfir Yehuda Levy and Yair Carmon},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
year={2025},
url={https://openreview.net/forum?id=VyjFOO9cFi}
}