NeurIPS 2020poster31 citations
Interpolation Technique to Speed Up Gradients Propagation in Neural ODEs
Talgat Daulbaev, Alexandr Katrutsa, Larisa Markeeva, Julia Gusak, Andrzej Cichocki, Ivan Oseledets
Abstract
We propose a simple interpolation-based method for the efficient approximation of gradients in neural ODE models. We compare it with reverse dynamic method (known in literature as “adjoint method”) to train neural ODEs on classification, density estimation and inference approximation tasks. We also propose a theoretical justification of our approach using logarithmic norm formalism. As a result, our method allows faster model training than the reverse dynamic method what was confirmed and validated by extensive numerical experiments for several standard benchmarks.
BibTeX
@inproceedings{NEURIPS2020_c24c6525,
author = {Daulbaev, Talgat and Katrutsa, Alexandr and Markeeva, Larisa and Gusak, Julia and Cichocki, Andrzej and Oseledets, Ivan},
booktitle = {Advances in Neural Information Processing Systems},
editor = {H. Larochelle and M. Ranzato and R. Hadsell and M.F. Balcan and H. Lin},
pages = {16689--16700},
publisher = {Curran Associates, Inc.},
title = {Interpolation Technique to Speed Up Gradients Propagation in Neural ODEs},
url = {https://proceedings.neurips.cc/paper_files/paper/2020/file/c24c65259d90ed4a19ab37b6fd6fe716-Paper.pdf},
volume = {33},
year = {2020}
}