2020
On the Noisy Gradient Descent that Generalizes as SGD
ICML 2020poster
The gradient noise of SGD is considered to play a central role in the observed strong generalization abilities of deep learning. While past studies confirm that the magnitude and the covariance structure of gradient noise are critical for regularization, it remains unclear whether or not the class o…