Andrew Hellicar

Efficient Orthogonal Parametrisation of Recurrent Neural Networks Using Householder Reflections

The problem of learning long-term dependencies in sequences using Recurrent Neural Networks (RNNs) is still a major challenge. Recent methods have been suggested to solve this problem by constraining the transition matrix to be unitary during training which ensures that its norm is equal to one and…