Prosodic Representation Learning and Contextual Sampling for Neural Text-to-Speech
Sri Karlapati, Ammar Abbas, Zack Hodari, Alexis Moinet, Arnaud Joly, Penny Karanasou, Thomas Drugman
Abstract
In this paper, we introduce Kathaka, a model trained with a novel two-stage training process for neural speech synthesis with contextually appropriate prosody. In Stage I, we learn a prosodic distribution at the sentence level from mel-spectrograms available during training. In Stage II, we propose a novel method to sample from this learnt prosodic distribution using the contextual information available in text. To do this, we use BERT on text, and graph-attention networks on parse trees extracted from text. We show a statistically significant relative improvement of 13.2% in naturalness over a strong baseline when compared to recordings. We also conduct an ablation study on variations of our sampling technique, and show a statistically significant improvement over the baseline in each case.
BibTeX
@inproceedings{icassp2021_prosodicrepresen,
title = {Prosodic Representation Learning and Contextual Sampling for Neural Text-to-Speech},
author = {Sri Karlapati and Ammar Abbas and Zack Hodari and Alexis Moinet and Arnaud Joly and Penny Karanasou and Thomas Drugman},
booktitle = {ICASSP 2021},
year = {2021}
}