Jezabel R Garcia

Fisher-Legendre (FishLeg) optimization of deep neural networks

Incorporating second-order gradient information (curvature) into optimization can dramatically reduce the number of iterations required to train machine learning models. In natural gradient descent, such information comes from the Fisher information matrix which yields a number of desirable properti…