Frederick Altrock

Momentum-SAM: Sharpness Aware Minimization without Computational Overhead

The recently proposed optimization algorithm for deep neural networks Sharpness Aware Minimization (SAM) suggests perturbing parameters before gradient calculation by a gradient ascent step to guide the optimization into parameter space regions of flat loss. While significant generalization improvem…