Zero-Shot Multi-Speaker Text-To-Speech with State-Of-The-Art Neural Speaker Embeddings

Erica Cooper; Cheng-I Lai; Yusuke Yasuda; Fuming Fang; Xin Wang; Nanxin Chen; Junichi Yamagishi

ICASSP 2020accepted0 citations

Zero-Shot Multi-Speaker Text-To-Speech with State-Of-The-Art Neural Speaker Embeddings

Erica Cooper, Cheng-I Lai, Yusuke Yasuda, Fuming Fang, Xin Wang, Nanxin Chen, Junichi Yamagishi

Abstract

While speaker adaptation for end-to-end speech synthesis using speaker embeddings can produce good speaker similarity for speakers seen during training, there remains a gap for zero-shot adaptation to unseen speakers. We investigate multi-speaker modeling for end-to-end text-to-speech synthesis and study the effects of different types of state-of-the-art neural speaker embeddings on speaker similarity for unseen speakers. Learnable dictionary encoding-based speaker embeddings with angular softmax loss can improve equal error rates over x-vectors in a speaker verification task; these embeddings also improve speaker similarity and naturalness for unseen speakers when used for zero-shot adaptation to new speakers in end-to-end speech synthesis.

BibTeX

@inproceedings{icassp2020_zeroshotmultispe,
  title = {Zero-Shot Multi-Speaker Text-To-Speech with State-Of-The-Art Neural Speaker Embeddings},
  author = {Erica Cooper and Cheng-I Lai and Yusuke Yasuda and Fuming Fang and Xin Wang and Nanxin Chen and Junichi Yamagishi},
  booktitle = {ICASSP 2020},
  year = {2020}
}