2021
Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics
ICLR 2021poster
Understanding the dynamics of neural network parameters during training is one of the key challenges in building a theoretical foundation for deep learning. A central obstacle is that the motion of a network in high-dimensional parameter space undergoes discrete finite steps along complex stochastic…