Cross-lingual Speech-based Tobi Label Generation Using Bidirectional Lstm

Abstract

In this paper we investigate the automatic generation of ToBI-style prosody labels. The work is motivated by the idea of using prosodic information to facilitate the automatic lexicon discovery for unseen and under-resourced languages for which sufficient training data is not available. Specifically, the prosodic boundaries are meant to serve as additional top-down information in the word segmentation step. To this end we attempt to apply the trained Japanese models cross-lingually on a language not seen in training (English). We generate break index labels, using only the speech signal as input, with no additional information given at test time in the form of transcripts or prior word segmentations. The labels are generated using bidirectional LSTMs trained on spontaneous Japanese speech. We evaluate the quality of these labels using established metrics, with an F1 score of 0.55 for cross-lingual prosodic break detection (given a tolerance of 80 ms).