Optimistic Meta-Gradients
Sebastian Flennerhag, Tom Zahavy, Brendan O'Donoghue, Hado van Hasselt, András György, Satinder Singh
Abstract
We study the connection between gradient-based meta-learning and convex optimisation. We observe that gradient descent with momentum is a special case of meta-gradients, and building on recent results in optimisation, we prove convergence rates for meta learning in the single task setting. While a meta-learned update rule can yield faster convergence up to constant factor, it is not sufficient for acceleration. Instead, some form of optimism is required. We show that optimism in meta-learning can be captured through the recently proposed Bootstrapped Meta-Gradient (Flennerhag et. al., 2022) method, providing deeper insight into its underlying mechanics.
BibTeX
@inproceedings{
flennerhag2023optimistic,
title={Optimistic Meta-Gradients},
author={Sebastian Flennerhag and Tom Zahavy and Brendan O'Donoghue and Hado van Hasselt and Andr{\'a}s Gy{\"o}rgy and Satinder Singh},
booktitle={Thirty-seventh Conference on Neural Information Processing Systems},
year={2023},
url={https://openreview.net/forum?id=ZmSg4f16uo}
}