Improving The Latency And Quality Of Cascaded Encoders
Tara N. Sainath, Yanzhang He, Arun Narayanan, Rami Botros, Weiran Wang, David Qiu, Chung-Cheng Chiu, Rohit Prabhavalkar
Abstract
In this paper, we explore reducing computational latency of the 2-pass cascaded encoder model [1]. Specifically, we experiment with reducing the size of the causal 1st-pass and adding capacity to the non-causal 2nd-pass, such that the overall latency can be reduced without loss of quality. In addition, we explore using a confidence model for deciding to stop 2nd-pass recognition if we are confident in the 1st-pass hypothesis. Overall, we are able to reduce latency by a factor of 1.7X, compared to the baseline cascaded encoder from [1]. Secondly, with the added capacity in the non-causal 2nd-pass, we find that we can improve WER by up to 7% relative using wav2vec and minimum word-error-rate (MWER) training.
BibTeX
@inproceedings{icassp2022_improvingthelate,
title = {Improving The Latency And Quality Of Cascaded Encoders},
author = {Tara N. Sainath and Yanzhang He and Arun Narayanan and Rami Botros and Weiran Wang and David Qiu and Chung-Cheng Chiu and Rohit Prabhavalkar and Alexander Gruenstein and Anmol Gulati and Bo Li and David Rybach and Emmanuel Guzman and Ian McGraw and James Qin and Krzysztof Choromanski and Qiao Liang and Robert David and Ruoming Pang and Shuo-Yiin Chang and Trevor Strohman and W. Ronny Huang and Wei Han and Yonghui Wu and Yu Zhang},
booktitle = {ICASSP 2022},
year = {2022}
}