← Search

Artem M Vysogorets

2 accepted papers

2024

Deconstructing the Goldilocks Zone of Neural Network Initialization

ICML 2024poster

The second-order properties of the training loss have a massive impact on the optimization dynamics of deep learning models. Fort & Scherlis (2019) discovered that a large excess of positive curvature and local convexity of the loss Hessian is associated with highly trainable initial points located…