3D Mouth Tracking from a Compact Microphone Array Co-Located with a camera
Xinyuan Qian, Alessio Xompero, Andrea Cavallaro, Alessio Brutti, Oswald Lanz, Maurizio Omologo
Abstract
We address the 3D audio-visual mouth tracking problem when using a compact platform with co-located audio-visual sensors, without a depth camera. In particular, we propose a multi-modal particle filter that combines a face detector and 3D hypothesis mapping to the image plane. The audio likelihood computation is assisted by video, which relies on a GCC-PHAT based acoustic map. By combining audio and video inputs, the proposed approach can cope with a reverberant and noisy environment, and can deal with situations when the person is occluded, outside the Field of View (FoV), or not facing the sensors. Experimental results show that the proposed tracker is accurate both in 3D and on the image plane.
BibTeX
@inproceedings{icassp2018_3dmouthtrackingf,
title = {3D Mouth Tracking from a Compact Microphone Array Co-Located with a camera},
author = {Xinyuan Qian and Alessio Xompero and Andrea Cavallaro and Alessio Brutti and Oswald Lanz and Maurizio Omologo},
booktitle = {ICASSP 2018},
year = {2018}
}