Itamar Menuhin-Gruman

Learning Low Dimensional State Spaces with Overparameterized Recurrent Neural Nets

Overparameterization in deep learning refers to settings where a trained Neural Network (NN) has representational capacity to fit the training data in many ways, some of which generalize well, while others do not. In the case of Recurrent Neural Networks (RNNs) there exists an additional layer of ov…