Fast-U2++: Fast and Accurate End-to-End Speech Recognition in Joint CTC/Attention Frames
Chengdong Liang, Xiao-Lei Zhang, Binbin Zhang, Di Wu, Shengqiang Li, Xingchen Song, Zhendong Peng, Fuping Pan
Abstract
Recently, the unified streaming and non-streaming two-pass (U2/U2++) end-to-end model for speech recognition has shown great performance in terms of streaming capability, accuracy and latency. In this paper, we present fast-U2++, an enhanced version of U2++ to further reduce partial latency. The core idea of fast-U2++ is to output partial results of the bottom layers in its encoder with a small chunk, while using a large chunk in the top layers of its encoder to compensate the performance degradation caused by the small chunk. More-over, we use knowledge distillation method to reduce the token emission latency. We present extensive experiments on Aishell-1 dataset. Experiments and ablation studies show that compared to U2++, fast-U2++ reduces model latency from 320ms to 80ms, and achieves a character error rate (CER) of 5.06% with a streaming setup.
BibTeX
@inproceedings{icassp2023_fastu2fastandacc,
title = {Fast-U2++: Fast and Accurate End-to-End Speech Recognition in Joint CTC/Attention Frames},
author = {Chengdong Liang and Xiao-Lei Zhang and Binbin Zhang and Di Wu and Shengqiang Li and Xingchen Song and Zhendong Peng and Fuping Pan},
booktitle = {ICASSP 2023},
year = {2023}
}