Tariq Daouda

Free Probability for predicting the performance of feed-forward fully connected neural networks

Gradient descent during the learning process of a neural network can be subject to many instabilities. The spectral density of the Jacobian is a key component for analyzing stability. Following the works of Pennington et al., such Jacobians are modeled using free multiplicative convolutions from Fre…