Continuous ultrasound based tongue movement video synthesis from speech
Jianrong Wang, Yalong Yang, Jianguo Wei, Ju Zhang
Abstract
The movement of tongue plays an important role in pronunciation. Visualizing the movement of tongue can improve speech intelligibility and also helps learning a second language. However, hardly any research has been investigated for this topic. In this paper, a framework to synthesize continuous ultrasound tongue movement video from speech is presented. Two different mapping methods are introduced as the most important parts of the framework. The objective evaluation and subjective opinions show that the Gaussian Mixture Model (GMM) based method has a better result for synthesizing static image and Vector Quantization (VQ) based method produces more stable continuous video. Meanwhile, the participants of evaluation state that the results of both methods are visual-understandable.
BibTeX
@inproceedings{icassp2016_continuousultras,
title = {Continuous ultrasound based tongue movement video synthesis from speech},
author = {Jianrong Wang and Yalong Yang and Jianguo Wei and Ju Zhang},
booktitle = {ICASSP 2016},
year = {2016}
}