Vit Fojtik

Conflicting Biases at the Edge of Stability: Norm versus Sharpness Regularization

The remarkable generalization properties of overparameterized networks are often attributed to implicit biases, such as norm minimization at small learning rates and low sharpness in the Edge-of-Stability regime. In this work, we argue that a comprehensive understanding of the generalization perform…