Luis A. Aguirre

Beyond exploding and vanishing gradients: analysing RNN training using attractors and smoothness

The exploding and vanishing gradient problem has been the major conceptual principle behind most architecture and training improvements in recurrent neural networks (RNNs) during the last decade. In this paper, we argue that this principle, while powerful, might need some refinement to explain rece…