NSV-TTS: Non-Speech Vocalization Modeling And Transfer In Emotional Text-To-Speech

Haitong Zhang; Xinyuan Yu; Yue Lin

ICASSP 2023accepted0 citations

NSV-TTS: Non-Speech Vocalization Modeling And Transfer In Emotional Text-To-Speech

Abstract

This paper addresses the problem of non-speech vocalization (NSV) modeling and transfer in emotional TTS. We propose an emotion TTS system (NSV-TTS) to model NSV and emotional speech. The model utilizes self-supervised learning to extract unsupervised linguistic units (ULUs) for NSV labeling and zero-shot NSV transfer. Furthermore, we propose token mixing and random masking to boost the performance. We evaluate the proposed method on various NSV types and emotion classes. The experimental results reveal that the proposed method performs well in the zero-shot NSV transfer task. Lastly, we conduct ablation studies to investigate the proposed method further.

BibTeX

@inproceedings{icassp2023_nsvttsnonspeechv,
  title = {NSV-TTS: Non-Speech Vocalization Modeling And Transfer In Emotional Text-To-Speech},
  author = {Haitong Zhang and Xinyuan Yu and Yue Lin},
  booktitle = {ICASSP 2023},
  year = {2023}
}