Investigations on sequence training of neural networks

Simon Wiesler; Pavel Golik; Ralf Schlüter; Hermann Ney

ICASSP 2015accepted0 citations

Investigations on sequence training of neural networks

Simon Wiesler, Pavel Golik, Ralf Schlüter, Hermann Ney

Abstract

In this paper we present an investigation of sequence-discriminative training of deep neural networks for automatic speech recognition. We evaluate different sequence-discriminative training criteria (MMI and MPE) and optimization algorithms (including SGD and Rprop) using the RASR toolkit. Further, we compare the training of the whole network with that of the output layer only. Technical details necessary for a robust training are studied, since there is no consensus yet on the ultimate training recipe. The investigation extends our previous work on training linear bottleneck networks from scratch showing the consistently positive effect of sequence training.

BibTeX

@inproceedings{icassp2015_investigationson,
  title = {Investigations on sequence training of neural networks},
  author = {Simon Wiesler and Pavel Golik and Ralf Schlüter and Hermann Ney},
  booktitle = {ICASSP 2015},
  year = {2015}
}