NeurIPS 2017poster127 citations
Integration Methods and Optimization Algorithms
Damien Scieur, Vincent Roulet, Francis Bach, Alexandre d'Aspremont
Abstract
We show that accelerated optimization methods can be seen as particular instances of multi-step integration schemes from numerical analysis, applied to the gradient flow equation. Compared with recent advances in this vein, the differential equation considered here is the basic gradient flow, and we derive a class of multi-step schemes which includes accelerated algorithms, using classical conditions from numerical analysis. Multi-step schemes integrate the differential equation using larger step sizes, which intuitively explains the acceleration phenomenon.
BibTeX
@inproceedings{NIPS2017_bf62768c,
author = {Scieur, Damien and Roulet, Vincent and Bach, Francis and d\textquotesingle Aspremont, Alexandre},
booktitle = {Advances in Neural Information Processing Systems},
editor = {I. Guyon and U. Von Luxburg and S. Bengio and H. Wallach and R. Fergus and S. Vishwanathan and R. Garnett},
pages = {},
publisher = {Curran Associates, Inc.},
title = {Integration Methods and Optimization Algorithms},
url = {https://proceedings.neurips.cc/paper_files/paper/2017/file/bf62768ca46b6c3b5bea9515d1a1fc45-Paper.pdf},
volume = {30},
year = {2017}
}