Teresa Dorszewski

How Redundant Is the Transformer Stack in Speech Representation Models?

Self-supervised speech representation models, particularly those leveraging transformer architectures, have demonstrated remarkable performance across various tasks such as speech recognition, speaker identification, and emotion detection. Recent studies on transformer models revealed high redundanc…