Luis Herrmann

Chaotic Dynamics are Intrinsic to Neural Network Training with SGD

With the advent of deep learning over the last decade, a considerable amount of effort has gone into better understanding and enhancing Stochastic Gradient Descent so as to improve the performance and stability of artificial neural network training. Active research fields in this area include exploi…