ICML 2022spotlight8 citations
Adaptive Accelerated (Extra-)Gradient Methods with Variance Reduction
Zijian Liu, Ta Duy Nguyen, Alina Ene, Huy Nguyen
Abstract
In this paper, we study the finite-sum convex optimization problem focusing on the general convex case. Recently, the study of variance reduced (VR) methods and their accelerated variants has made exciting progress. However, the step size used in the existing VR algorithms typically depends on the smoothness parameter, which is often unknown and requires tuning in practice. To address this problem, we propose two novel adaptive VR algorithms:
BibTeX
@InProceedings{pmlr-v162-liu22o,
title = {Adaptive Accelerated ({E}xtra-){G}radient Methods with Variance Reduction},
author = {Liu, Zijian and Nguyen, Ta Duy and Ene, Alina and Nguyen, Huy},
booktitle = {Proceedings of the 39th International Conference on Machine Learning},
pages = {13947--13994},
year = {2022},
editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan},
volume = {162},
series = {Proceedings of Machine Learning Research},
month = {17--23 Jul},
publisher = {PMLR},
pdf = {https://proceedings.mlr.press/v162/liu22o/liu22o.pdf},
url = {https://proceedings.mlr.press/v162/liu22o.html},
abstract = {In this paper, we study the finite-sum convex optimization problem focusing on the general convex case. Recently, the study of variance reduced (VR) methods and their accelerated variants has made exciting progress. However, the step size used in the existing VR algorithms typically depends on the smoothness parameter, which is often unknown and requires tuning in practice. To address this problem, we propose two novel adaptive VR algorithms: