Philippe Jacquet

Quasi Black Hole Effect of Gradient Descent in Large Dimension: Consequence on Neural Network Learning

The gradient descent to a local minimum is the key ingredient of deep neural networks learning techniques. We consider a function L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</sub> (.) in dimension n with a random set of m absolute minima. When l…