Massive End-to-end Speech Recognition Models with Time Reduction
Weiran Wang, Rohit Prabhavalkar, Haozhe Shan, Zhong Meng, Dongseong Hwang, Qiujia Li, Khe Chai Sim, Bo Li
Abstract
We investigate massive end-to-end automatic speech recognition (ASR) models with efficiency improvements achieved by time reduction. The encoders of our models use the neural architecture of Google’s universal speech model (USM), with additional funnel pooling layers to significantly reduce the frame rate and speed up training and inference. We also explore a few practical methods to mitigate potential accuracy loss due to time reduction, while enjoying most efficiency gain. Our methods are demonstrated to work with both Connectionist Temporal Classification (CTC) and RNN-Transducer (RNN-T), with up to 2B model parameters, and over two domains. For a large-scale voice search recognition task, we perform extensive studies on vocabulary size, time reduction strategy, and its generalization performance on long-form test sets, and show that a 900M RNN-T is very tolerant to severe time reduction, with as low encoder output frame rate as 640ms. We also provide ablation studies on the Librispeech benchmark for important training hyperparameters and architecture designs, in training 600M RNN-T models at the frame rate of 160ms.
BibTeX
@inproceedings{wang-etal-2024-massive,
title = "Massive End-to-end Speech Recognition Models with Time Reduction",
author = "Wang, Weiran and
Prabhavalkar, Rohit and
Shan, Haozhe and
Meng, Zhong and
Hwang, Dongseong and
Li, Qiujia and
Sim, Khe Chai and
Li, Bo and
Qin, James and
Cai, Xingyu and
Stooke, Adam and
Zheng, Chengjian and
He, Yanzhang and
Sainath, Tara and
Moreno Mengibar, Pedro",
editor = "Duh, Kevin and
Gomez, Helena and
Bethard, Steven",
booktitle = "Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)",
month = jun,
year = "2024",
address = "Mexico City, Mexico",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2024.naacl-long.344/",
doi = "10.18653/v1/2024.naacl-long.344",
pages = "6206--6217"
}