Adaptive gradient descent on Riemannian manifolds and its applications to Gaussian variational inference
Jiyoung Park, Jaewook J. Suh, Bofan Wang, Anirban Bhattacharya, Shiqian Ma
Abstract
We propose RAdaGD, a novel family of adaptive gradient descent methods on general Riemannian manifolds. RAdaGD adapts the step size parameter without line search, and includes instances that achieve a non-ergodic convergence guarantee, $f(x_k) - f(x_\star) \le \mathcal{O}(1/k)$, under local geodesic smoothness and generalized geodesic convexity. A core application of RAdaGD is Gaussian Variational Inference, where our method provides the first convergence guarantee in the absence of $L$-smoothness of the target log-density, under additional technical assumptions. We also investigate the empirical performance of RAdaGD in numerical simulations and demonstrate its competitiveness in comparison to existing algorithms.
BibTeX
@inproceedings{
park2026adaptive,
title={Adaptive gradient descent on Riemannian manifolds and its applications to Gaussian variational inference},
author={Jiyoung Park and Jaewook J. Suh and Bofan Wang and Anirban Bhattacharya and Shiqian Ma},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026},
url={https://openreview.net/forum?id=2TTQjRkgFn}
}