NeurIPS 2021oral37 citations
Framing RNN as a kernel method: A neural ODE approach
Adeline Fermanian, Pierre Marion, Jean-Philippe Vert, Gérard Biau
Abstract
Building on the interpretation of a recurrent neural network (RNN) as a continuous-time neural differential equation, we show, under appropriate conditions, that the solution of a RNN can be viewed as a linear function of a specific feature set of the input sequence, known as the signature. This connection allows us to frame a RNN as a kernel method in a suitable reproducing kernel Hilbert space. As a consequence, we obtain theoretical guarantees on generalization and stability for a large class of recurrent networks. Our results are illustrated on simulated datasets.
recurrent neural networksneural ODEkernel methodtheory of deep learninggeneralization bounds
BibTeX
@inproceedings{
fermanian2021framing,
title={Framing {RNN} as a kernel method: A neural {ODE} approach},
author={Adeline Fermanian and Pierre Marion and Jean-Philippe Vert and G{\'e}rard Biau},
booktitle={Advances in Neural Information Processing Systems},
editor={A. Beygelzimer and Y. Dauphin and P. Liang and J. Wortman Vaughan},
year={2021},
url={https://openreview.net/forum?id=QT9ulkiN-LX}
}