NeurIPS 2023poster4 citations
On the Convergence to a Global Solution of Shuffling-Type Gradient Algorithms
Abstract
Stochastic gradient descent (SGD) algorithm is the method of choice in many machine learning tasks thanks to its scalability and efficiency in dealing with large-scale problems. In this paper, we focus on the shuffling version of SGD which matches the mainstream practical heuristics. We show the convergence to a global solution of shuffling SGD for a class of non-convex functions under over-parameterized settings. Our analysis employs more relaxed non-convex assumptions than previous literature. Nevertheless, we maintain the desired computational complexity as shuffling SGD has achieved in the general convex setting.
stochastic gradientshuffling type gradient methodglobal convergence
BibTeX
@inproceedings{
nguyen2023on,
title={On the Convergence to a Global Solution of Shuffling-Type Gradient Algorithms},
author={Lam M. Nguyen and Trang H. Tran},
booktitle={Thirty-seventh Conference on Neural Information Processing Systems},
year={2023},
url={https://openreview.net/forum?id=Nr1XSeDzpn}
}