Text/Speech-Driven Full-Body Animation
Wenlin Zhuang, Jinwei Qi, Peng Zhang, Bang Zhang, Ping Tan
Abstract
Due to the increasing demand in films and games, synthesizing 3D avatar animation has attracted much attention recently. In this work, we present a production-ready text/speech-driven full-body animation synthesis system. Given the text and corresponding speech, our system synthesizes face and body animations simultaneously, which are then skinned and rendered to obtain a video stream output. We adopt a learning-based approach for synthesizing facial animation and a graph-based approach to animate the body, which generates high-quality avatar animation efficiently and robustly. Our results demonstrate the generated avatar animations are realistic, diverse and highly text/speech-correlated.
BibTeX
@inproceedings{ijcai2022p863,
title = {Text/Speech-Driven Full-Body Animation},
author = {Zhuang, Wenlin and Qi, Jinwei and Zhang, Peng and Zhang, Bang and Tan, Ping},
booktitle = {Proceedings of the Thirty-First International Joint Conference on
Artificial Intelligence, {IJCAI-22}},
publisher = {International Joint Conferences on Artificial Intelligence Organization},
editor = {Lud De Raedt},
pages = {5956--5959},
year = {2022},
month = {7},
note = {Demo Track},
doi = {10.24963/ijcai.2022/863},
url = {https://doi.org/10.24963/ijcai.2022/863},
}