NeurIPS 2023poster4 citations
A Variational Perspective on High-Resolution ODEs
Hoomaan Maskan, Konstantinos C. Zygalakis, Alp Yurtsever
Abstract
We consider unconstrained minimization of smooth convex functions. We propose a novel variational perspective using forced Euler-Lagrange equation that allows for studying high-resolution ODEs. Through this, we obtain a faster convergence rate for gradient norm minimization using Nesterov's accelerated gradient method. Additionally, we show that Nesterov's method can be interpreted as a rate-matching discretization of an appropriately chosen high-resolution ODE. Finally, using the results from the new variational perspective, we propose a stochastic method for noisy gradients. Several numerical experiments compare and illustrate our stochastic algorithm with state of the art methods.
Nesterov's accelerated gradientgradient descentLyapunov functiongradient norm minimizationrate-matchingstochastic variance reductionstochastic gradient descentnoisy gradient
BibTeX
@inproceedings{
maskan2023a,
title={A Variational Perspective on High-Resolution {ODE}s},
author={Hoomaan Maskan and Konstantinos C. Zygalakis and Alp Yurtsever},
booktitle={Thirty-seventh Conference on Neural Information Processing Systems},
year={2023},
url={https://openreview.net/forum?id=TXq8PCRSoY}
}