Estelle Massart

Coordinate Descent on the Orthogonal Group for Recurrent Neural Network Training

We address the poor scalability of learning algorithms for orthogonal recurrent neural networks via the use of stochastic coordinate descent on the orthogonal group, leading to a cost per iteration that increases linearly with the number of recurrent states. This contrasts with the cubic dependency…