Riemannian Stochastic Recursive Momentum Method for non-Convex Optimization

Andi Han; Junbin Gao

IJCAI 2021poster12 citations

Riemannian Stochastic Recursive Momentum Method for non-Convex Optimization

Abstract

We propose a stochastic recursive momentum method for Riemannian non-convex optimization that achieves a nearly-optimal complexity to find epsilon-approximate solution with one sample. The new algorithm requires one-sample gradient evaluations per iteration and does not require restarting with a large batch gradient, which is commonly used to obtain a faster rate. Extensive experiment results demonstrate the superiority of the proposed algorithm. Extensions to nonsmooth and constrained optimization settings are also discussed.

Machine Learning: Online Learning

BibTeX

@inproceedings{ijcai2021p345,
  title     = {Riemannian Stochastic Recursive Momentum Method for non-Convex Optimization},
  author    = {Han, Andi and Gao, Junbin},
  booktitle = {Proceedings of the Thirtieth International Joint Conference on
               Artificial Intelligence, {IJCAI-21}},
  publisher = {International Joint Conferences on Artificial Intelligence Organization},
  editor    = {Zhi-Hua Zhou},
  pages     = {2505--2511},
  year      = {2021},
  month     = {8},
  note      = {Main Track},
  doi       = {10.24963/ijcai.2021/345},
  url       = {https://doi.org/10.24963/ijcai.2021/345},
}