2020
An Empirical Study of Transformer-Based Neural Language Model Adaptation
ICASSP 2020accepted
We explore two adaptation approaches of deep Transformer based neural language models (LMs) for automatic speech recognition. The first approach is a pretrain-finetune framework, where we first pretrain a Transformer LM on a large-scale text corpus from scratch and then adapt it to relatively small…