Adaptive Decoding for Efficient Automatic Speech Recognition
Xiangnan Ma, Peizhuo Liu, Yuhao Zhang, Kaiqi Kou, Chenghao Gao, Tong Xiao, Jingbo Zhu
Abstract
The latency and computation demand of End-to-end (E2E) automatic speech recognition (ASR) models hinder their deployment on lightweight devices. Despite there are many methods proposed for efficiency, the computational burden of the output layer with a large vocabulary is still a major challenge for decoders. In this paper, we propose an adaptive decoding method (ADD) to reduce the latency. Based on the vocal features of words like phonemes or speech units, we cluster the original vocabulary into small sets, allowing the model to inference more efficiently with a smaller search space. Experimental results demonstrate that our method significantly reduces the calculation FLOPs while maintaining performance. We also provide a deeper understanding of speech units from the perspective of phonemes.
BibTeX
@inproceedings{icassp2025_adaptivedecoding,
title = {Adaptive Decoding for Efficient Automatic Speech Recognition},
author = {Xiangnan Ma and Peizhuo Liu and Yuhao Zhang and Kaiqi Kou and Chenghao Gao and Tong Xiao and Jingbo Zhu},
booktitle = {ICASSP 2025},
year = {2025}
}