Riemannian Stochastic Recursive Gradient Algorithm
Hiroyuki Kasai, Hiroyuki Sato, Bamdev Mishra
Abstract
Stochastic variance reduction algorithms have recently become popular for minimizing the average of a large, but finite number of loss functions on a Riemannian manifold. The present paper proposes a Riemannian stochastic recursive gradient algorithm (R-SRG), which does not require the inverse of retraction between two distant iterates on the manifold. Convergence analyses of R-SRG are performed on both retraction-convex and non-convex functions under computationally efficient retraction and vector transport operations. The key challenge is analysis of the influence of vector transport along the retraction curve. Numerical evaluations reveal that R-SRG competes well with state-of-the-art Riemannian batch and stochastic gradient algorithms.
BibTeX
@InProceedings{pmlr-v80-kasai18a,
title = {{R}iemannian Stochastic Recursive Gradient Algorithm},
author = {Kasai, Hiroyuki and Sato, Hiroyuki and Mishra, Bamdev},
booktitle = {Proceedings of the 35th International Conference on Machine Learning},
pages = {2516--2524},
year = {2018},
editor = {Dy, Jennifer and Krause, Andreas},
volume = {80},
series = {Proceedings of Machine Learning Research},
month = {10--15 Jul},
publisher = {PMLR},
pdf = {http://proceedings.mlr.press/v80/kasai18a/kasai18a.pdf},
url = {https://proceedings.mlr.press/v80/kasai18a.html},
abstract = {Stochastic variance reduction algorithms have recently become popular for minimizing the average of a large, but finite number of loss functions on a Riemannian manifold. The present paper proposes a Riemannian stochastic recursive gradient algorithm (R-SRG), which does not require the inverse of retraction between two distant iterates on the manifold. Convergence analyses of R-SRG are performed on both retraction-convex and non-convex functions under computationally efficient retraction and vector transport operations. The key challenge is analysis of the influence of vector transport along the retraction curve. Numerical evaluations reveal that R-SRG competes well with state-of-the-art Riemannian batch and stochastic gradient algorithms.}
}