Prediction of Voicing and the F0 Contour from Electromagnetic Articulography Data for Articulation-to-Speech Synthesis

Simon Stone; Philipp Schmidt; Peter Birkholz

ICASSP 2020accepted0 citations

Prediction of Voicing and the F0 Contour from Electromagnetic Articulography Data for Articulation-to-Speech Synthesis

Simon Stone, Philipp Schmidt, Peter Birkholz

Abstract

Articulation-to-speech synthesis based solely on supraglottal articulation requires some sort of intonation control. This paper examines to what extent the f 0 contour of an utterance can be predicted from such supraglottal articulation data. To that end, three groups of machine learning models (support vector machines, kernel ridge regression and neural networks) were trained and evaluated on the mngu0speech corpus containing synchronous articulatory and audio data. The best voiced/unvoiced/silence classification rates were achieved by a deep neural network with two hidden layers: 85.8 % with no look-ahead (important for on-line applications) and 86 % with a look-ahead of 50 ms. The best f 0 prediction model without look-ahead scored a root-mean-square error (RMSE) (when compared to the original f 0 contours) of 10.4 Hz using a neural network with one hidden layer, while the best prediction with a look-ahead of 50 ms was attained by kernel ridge regression and an RMSE of 10.3 Hz. The predicted f 0 contours were also subjectively evaluated in a listening test by manipulating the f 0 of the original speech files using PRAAT. The results are consistent with the objective evaluation.

BibTeX

@inproceedings{icassp2020_predictionofvoic,
  title = {Prediction of Voicing and the F0 Contour from Electromagnetic Articulography Data for Articulation-to-Speech Synthesis},
  author = {Simon Stone and Philipp Schmidt and Peter Birkholz},
  booktitle = {ICASSP 2020},
  year = {2020}
}