Continuous-time Analysis of Anchor Acceleration
Jaewook J. Suh, Jisun Park, Ernest K. Ryu
Abstract
Recently, the anchor acceleration, an acceleration mechanism distinct from Nesterov's, has been discovered for minimax optimization and fixed-point problems, but its mechanism is not understood well, much less so than Nesterov acceleration. In this work, we analyze continuous-time models of anchor acceleration. We provide tight, unified analyses for characterizing the convergence rate as a function of the anchor coefficient $\beta(t)$, thereby providing insight into the anchor acceleration mechanism and its accelerated $\mathcal{O}(1/k^2)$-convergence rate. Finally, we present an adaptive method inspired by the continuous-time analyses and establish its effectiveness through theoretical analyses and experiments.
BibTeX
@inproceedings{
suh2023continuoustime,
title={Continuous-time Analysis of Anchor Acceleration},
author={Jaewook J. Suh and Jisun Park and Ernest K. Ryu},
booktitle={Thirty-seventh Conference on Neural Information Processing Systems},
year={2023},
url={https://openreview.net/forum?id=rN99gLCBe4}
}