Jintao Tan

Uni-Dubbing: Zero-Shot Speech Synthesis from Visual Articulation

In the field of speech synthesis, there is a growing emphasis on employing multimodal speech to enhance robustness. A key challenge in this area is the scarcity of datasets that pair audio with corresponding video. We employ a methodology that incorporates modality alignment during the pre-training…