AISTATS 2021poster7 citations
Generalization of Quasi-Newton Methods: Application to Robust Symmetric Multisecant Updates
Damien Scieur, Lewis Liu, Thomas Pumir, Nicolas Boumal
Abstract
Quasi-Newton (qN) techniques approximate the Newton step by estimating the Hessian using the so-called secant equations. Some of these methods compute the Hessian using several secant equations but produce non-symmetric updates. Other quasi-Newton schemes, such as BFGS, enforce symmetry but cannot satisfy more than one secant equation. We propose a new type of quasi-Newton symmetric update using several secant equations in a least-squares sense. Our approach generalizes and unifies the design of quasi-Newton updates and satisfies provable robustness guarantees.
BibTeX
@InProceedings{pmlr-v130-scieur21a,
title = { Generalization of Quasi-Newton Methods: Application to Robust Symmetric Multisecant Updates },
author = {Scieur, Damien and Liu, Lewis and Pumir, Thomas and Boumal, Nicolas},
booktitle = {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics},
pages = {550--558},
year = {2021},
editor = {Banerjee, Arindam and Fukumizu, Kenji},
volume = {130},
series = {Proceedings of Machine Learning Research},
month = {13--15 Apr},
publisher = {PMLR},
pdf = {http://proceedings.mlr.press/v130/scieur21a/scieur21a.pdf},
url = {https://proceedings.mlr.press/v130/scieur21a.html},
abstract = { Quasi-Newton (qN) techniques approximate the Newton step by estimating the Hessian using the so-called secant equations. Some of these methods compute the Hessian using several secant equations but produce non-symmetric updates. Other quasi-Newton schemes, such as BFGS, enforce symmetry but cannot satisfy more than one secant equation. We propose a new type of quasi-Newton symmetric update using several secant equations in a least-squares sense. Our approach generalizes and unifies the design of quasi-Newton updates and satisfies provable robustness guarantees. }
}