SkipW: Resource Adaptable RNN with Strict Upper Computational Limit
Tsiry Mayet, Anne Lambert, Pascal Leguyadec, Francoise Le Bolzer, François Schnitzler
Abstract
We introduce Skip-Window, a method to allow recurrent neural networks (RNNs) to trade off accuracy for computational cost during the analysis of a sequence. Similarly to existing approaches, Skip-Window extends existing RNN cells by adding a mechanism to encourage the model to process fewer inputs. Unlike existing approaches, Skip-Window is able to respect a strict computational budget, making this model more suitable for limited hardware. We evaluate this approach on two datasets: a human activity recognition task and adding task. Our results show that Skip-Window is able to exceed the accuracy of existing approaches for a lower computational cost while strictly limiting said cost.
BibTeX
@inproceedings{
mayet2021skipw,
title={SkipW: Resource Adaptable {\{}RNN{\}} with Strict Upper Computational Limit},
author={Tsiry Mayet and Anne Lambert and Pascal Leguyadec and Francoise Le Bolzer and Fran{\c{c}}ois Schnitzler},
booktitle={International Conference on Learning Representations},
year={2021},
url={https://openreview.net/forum?id=2CjEVW-RGOJ}
}