ICML 2019oral23 citations
Simple Stochastic Gradient Methods for Non-Smooth Non-Convex Regularized Optimization
Abstract
Our work focuses on stochastic gradient methods for optimizing a smooth non-convex loss function with a non-smooth non-convex regularizer. Research on this class of problem is quite limited, and until recently no non-asymptotic convergence results have been reported. We present two simple stochastic gradient algorithms, for finite-sum and general stochastic optimization problems, which have superior convergence complexities compared to the current state-of-the-art. We also compare our algorithms’ performance in practice for empirical risk minimization.
BibTeX
@InProceedings{pmlr-v97-metel19a,
title = {Simple Stochastic Gradient Methods for Non-Smooth Non-Convex Regularized Optimization},
author = {Metel, Michael and Takeda, Akiko},
booktitle = {Proceedings of the 36th International Conference on Machine Learning},
pages = {4537--4545},
year = {2019},
editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan},
volume = {97},
series = {Proceedings of Machine Learning Research},
month = {09--15 Jun},
publisher = {PMLR},
pdf = {http://proceedings.mlr.press/v97/metel19a/metel19a.pdf},
url = {https://proceedings.mlr.press/v97/metel19a.html},
abstract = {Our work focuses on stochastic gradient methods for optimizing a smooth non-convex loss function with a non-smooth non-convex regularizer. Research on this class of problem is quite limited, and until recently no non-asymptotic convergence results have been reported. We present two simple stochastic gradient algorithms, for finite-sum and general stochastic optimization problems, which have superior convergence complexities compared to the current state-of-the-art. We also compare our algorithms’ performance in practice for empirical risk minimization.}
}