Audio Sparse-Transformer for Speech Classification

Abstract

State-of-the-art models employ transformers for audio classification [1]. However, the computational complexity of transformers significantly increases due to the computation of self-attention over long sequences. Therefore, we propose an approach that replaces the long sequence of time frames with a short sequence of tokens. These tokens are obtained by identifying significant regions in the spectrogram, sampling within them, and decoding them for inputs to the transformer. The results of our experiments show that our sparse-transformer model archives an accuracy of 96.88% on the Google Speech Command V2 dataset, while maintaining very low computational costs. The computational cost (FLOPs) is reduced by 85.25% compared to a state-of-the-art model, EAT-S [1].