A Comparison of Semi-Supervised Learning Techniques for Streaming ASR at Scale
Cal Peyser, Michael Picheny, Kyunghyun Cho, Rohit Prabhavalkar, W. Ronny Huang, Tara N. Sainath
Abstract
Unpaired text and audio injection have emerged as dominant methods for improving ASR performance in the absence of a large labeled corpus. However, little guidance exists on deploying these methods to improve production ASR systems that are trained on very large supervised corpora and with realistic requirements like a constrained model size and CPU budget, streaming capability, and a rich lattice for rescoring and for downstream NLU tasks. In this work, we compare three state-of-the-art semi-supervised methods encompassing both unpaired text and audio as well as several of their combinations in a controlled setting using joint training. We find that in our setting these methods offer many improvements beyond raw WER, including substantial gains in tail-word WER, decoder computation during inference, and lattice density.
BibTeX
@inproceedings{icassp2023_acomparisonofsem,
title = {A Comparison of Semi-Supervised Learning Techniques for Streaming ASR at Scale},
author = {Cal Peyser and Michael Picheny and Kyunghyun Cho and Rohit Prabhavalkar and W. Ronny Huang and Tara N. Sainath},
booktitle = {ICASSP 2023},
year = {2023}
}