An Attention-Based Joint Acoustic and Text on-Device End-To-End Model
Tara N. Sainath, Ruoming Pang, Ron J. Weiss, Yanzhang He, Chung-Cheng Chiu, Trevor Strohman
Abstract
Recently, we introduced a two-pass on-device end-to-end (E2E) speech recognition model, which runs RNN-T in the first-pass and then rescores/redecodes the result using a noncausal Listen, Attend and Spell (LAS) decoder. This on-device model obtained similar performance to a state-of-the-art conventional model. However, like many E2E models, it suffers from being trained only on supervised audio-text pairs and thus performs poorly on rare words compared to a conventional model which incorporates a language model trained on a much larger text corpus. In this work, we introduce a joint acoustic and text decoder (JATD) into the LAS decoder, which makes it possible to incorporate a much larger text corpus into training. We find that the JATD model obtains in a 3-10% relative improvement in WER compared to a LAS decoder trained only on supervised audio-text pairs across a variety of proper noun test sets.
BibTeX
@inproceedings{icassp2020_anattentionbased,
title = {An Attention-Based Joint Acoustic and Text on-Device End-To-End Model},
author = {Tara N. Sainath and Ruoming Pang and Ron J. Weiss and Yanzhang He and Chung-Cheng Chiu and Trevor Strohman},
booktitle = {ICASSP 2020},
year = {2020}
}