Phoneme-Level Bert for Enhanced Prosody of Text-To-Speech with Grapheme Predictions

Yinghao Aaron Li; Cong Han; Xilin Jiang; Nima Mesgarani

ICASSP 2023accepted0 citations

Phoneme-Level Bert for Enhanced Prosody of Text-To-Speech with Grapheme Predictions

Yinghao Aaron Li, Cong Han, Xilin Jiang, Nima Mesgarani

Abstract

Large-scale pre-trained language models have been shown to be helpful in improving the naturalness of text-to-speech (TTS) models by enabling them to produce more naturalistic prosodic patterns. However, these models are usually word-level or sup-phoneme-level and jointly trained with phonemes, making them inefficient for the downstream TTS task where only phonemes are needed. In this work, we propose a phoneme-level BERT (PL-BERT) with a pretext task of predicting the corresponding graphemes along with the regular masked phoneme predictions. Subjective evaluations show that our phoneme-level BERT encoder has significantly improved the mean opinion scores (MOS) of rated naturalness of synthesized speech compared with the state-of-the-art (SOTA) StyleTTS baseline on out-of-distribution (OOD) texts.

BibTeX

@inproceedings{icassp2023_phonemelevelbert,
  title = {Phoneme-Level Bert for Enhanced Prosody of Text-To-Speech with Grapheme Predictions},
  author = {Yinghao Aaron Li and Cong Han and Xilin Jiang and Nima Mesgarani},
  booktitle = {ICASSP 2023},
  year = {2023}
}