Tsun-Yat Leung

Conformer-Based Speech Recognition with Linear Nyström Attention and Rotary Position Embedding

Self-attention has become an important component for end-to-end (E2E) automatic speech recognition (ASR). Recently, Convolution-augmented Transformer (Conformer) with relative positional encoding (RPE) achieved state-of-the-art performance. However, the computational and memory complexity of self-at…