Karlis Freivalds — accepted AI-conference papers

2021

Residual Shuffle-Exchange Networks for Fast Processing of Long Sequences

Andis Draguns, Emīls Ozoliņš, Agris Šostaks, Matīss Apinis, Karlis Freivalds

AAAI 2021technical

Attention is a commonly used mechanism in sequence processing, but it is of O(n^2) complexity which prevents its application to long sequences. The recently introduced neural Shuffle-Exchange network offers a computation-efficient alternative, enabling the modelling of long-range dependencies in O(n…

Cited by 7 Source ↗PDF ↗code ↗Scholar ↗

2019

Neural Shuffle-Exchange Networks - Sequence Processing in O(n log n) Time

Karlis Freivalds, Emīls Ozoliņš, Agris Šostaks

NeurIPS 2019poster

A key requirement in sequence to sequence processing is the modeling of long range dependencies. To this end, a vast majority of the state-of-the-art models use attention mechanism which is of O(n^2) complexity that leads to slow execution for long sequences.

Cited by 19 Source ↗PDF ↗code ↗Scholar ↗