2020
Lookahead Converges to Stationary Points of Smooth Non-convex Functions
ICASSP 2020accepted
The Lookahead optimizer [Zhang et al., 2019] was recently proposed and demonstrated to improve performance of stochastic first-order methods for training deep neural networks. Lookahead can be viewed as a two time-scale algorithm, where the fast dynamics (inner optimizer) determine a search directio…