Samuel L. Smith and Quoc V. Le

A Bayesian Perspective on Generalization and Stochastic Gradient Descent

We consider two questions at the heart of machine learning; how can we predict if a minimum will generalize to the test set, and why does stochastic gradient descent find minima that generalize well? Our work responds to \citet{zhang2016understanding}, who showed deep neural networks can easily memo…