Large-Scale Unsupervised Pre-Training for End-to-End Spoken Language Understanding

Pengwei Wang; Liangchen Wei; Yong Cao; Jinghui Xie; Zaiqing Nie

ICASSP 2020accepted0 citations

Large-Scale Unsupervised Pre-Training for End-to-End Spoken Language Understanding

Pengwei Wang, Liangchen Wei, Yong Cao, Jinghui Xie, Zaiqing Nie

Abstract

End-to-end Spoken Language Understanding (SLU) is proposed to infer the semantic meaning directly from audio features without intermediate text representation. In this paper, we explore unsupervised pre-training for End-to-end SLU models by learning representations from large-scale raw audios. The pre-trained model preserves semantic features which benefit the downstream SLU tasks as the learned model weights are further fine-tuned on the task specific training data. Our approach out-perform the state-of-the-art end-to-end SLU system with over 18.33% error reduction.

BibTeX

@inproceedings{icassp2020_largescaleunsupe,
  title = {Large-Scale Unsupervised Pre-Training for End-to-End Spoken Language Understanding},
  author = {Pengwei Wang and Liangchen Wei and Yong Cao and Jinghui Xie and Zaiqing Nie},
  booktitle = {ICASSP 2020},
  year = {2020}
}