Simultaneous Speech Recognition and Acoustic Event Detection Using an LSTM-CTC Acoustic Model and a WFST Decoder

Abstract

This paper proposes a new approach to simultaneous speech recognition and acoustic event detection of spontaneous speech based on one-pass decoding without rescoring. In this approach, an LSTM (long short-term memory) acoustic model outputs probabilities of acoustic event symbols, such as filler symbols and word fragment symbols, as well as probabilities of phonetic symbols. Then a WFST (weighted finite state transducer) decoder detects fillers based on a filler confidence score calculated by the ratio of the number of filler symbols to the number of phonetic symbols in each word. The WFST decoder also detects word fragments using a phonetic symbol loop followed by a path accepting the word fragment symbol, which is attached to a WFST of a lexicon. Experimental results show that precision and recall rates of filler detection can be controlled by the filler confidence score, and word fragments can be detected without registering all possible word fragments to the lexicon.