Improvements to N-gram Language Model Using Text Generated from Neural Language Model
Masayuki Suzuki, Nobuyasu Itoh, Tohru Nagano, Gakuto Kurata, Samuel Thomas
Abstract
Although neural language models have emerged, n-gram language models are still used for many speech recognition tasks. This paper proposes four methods to improve n-gram language models using text generated from a recurrent neural network language model (RNNLM). First, we use multiple RNNLMs from different domains instead of a single RNNLM. The final n-gram language model is obtained by interpolating generated n-gram models from each domain. Second, we use subwords instead of words for RNNLM to reduce the out-of-vocabulary rate. Third, we generate text templates using an RNNLM for template-based data augmentation for named entities. Fourth, we use both forward RNNLM and backward RNNLM to generate text. We found that these four methods improved performance of speech recognition up to 4% relative in various tasks.
BibTeX
@inproceedings{icassp2019_improvementstong,
title = {Improvements to N-gram Language Model Using Text Generated from Neural Language Model},
author = {Masayuki Suzuki and Nobuyasu Itoh and Tohru Nagano and Gakuto Kurata and Samuel Thomas},
booktitle = {ICASSP 2019},
year = {2019}
}