Ivana Kajić

Legendre Memory Units: Continuous-Time Representation in Recurrent Neural Networks

We propose a novel memory cell for recurrent neural networks that dynamically maintains information across long windows of time using relatively few resources. The Legendre Memory Unit~(LMU) is mathematically derived to orthogonalize its continuous-time history -- doing so by solving $d$ coupled ord…