Regularization of context-dependent deep neural networks with context-independent multi-task training

Peter Bell; Steve Renals

ICASSP 2015accepted0 citations

Regularization of context-dependent deep neural networks with context-independent multi-task training

Abstract

The use of context-dependent targets has become standard in hybrid DNN systems for automatic speech recognition. However, we argue that despite the use of state-tying, optimising to context-dependent targets can lead to over-fitting, and that discriminating between arbitrary tied context-dependent targets may not be optimal. We propose a multitask learning method where the network jointly predicts context-dependent and monophone targets. We evaluate the method on a large-vocabulary lecture recognition task and show that it yields relative improvements of 3-10% over baseline systems.

BibTeX

@inproceedings{icassp2015_regularizationof,
  title = {Regularization of context-dependent deep neural networks with context-independent multi-task training},
  author = {Peter Bell and Steve Renals},
  booktitle = {ICASSP 2015},
  year = {2015}
}