The First Multimodal Information Based Speech Processing (Misp) Challenge: Data, Tasks, Baselines And Results
Hang Chen, Hengshun Zhou, Jun Du, Chin-Hui Lee, Jingdong Chen, Shinji Watanabe, Sabato Marco Siniscalchi, Odette Scharenborg
Abstract
In this paper we discuss the rational of the Multi-model Information based Speech Processing (MISP) Challenge, and provide a detailed description of the data recorded, the two evaluation tasks and the corresponding baselines, followed by a summary of submitted systems and evaluation results. The MISP Challenge aims at tack-ling speech processing tasks in different scenarios by introducing information about an additional modality (e.g., video, or text), which will hopefully lead to better environmental and speaker robustness in realistic applications. In the first MISP challenge, two bench-mark datasets recorded in a real-home TV room with two reproducible open-source baseline systems have been released to promote research in audio-visual wake word spotting (AVWWS) and audio-visual speech recognition (AVSR). To our knowledge, MISP is the first open evaluation challenge to tackle real-world issues of AVWWS and AVSR in the home TV scenario.
BibTeX
@inproceedings{icassp2022_thefirstmultimod,
title = {The First Multimodal Information Based Speech Processing (Misp) Challenge: Data, Tasks, Baselines And Results},
author = {Hang Chen and Hengshun Zhou and Jun Du and Chin-Hui Lee and Jingdong Chen and Shinji Watanabe and Sabato Marco Siniscalchi and Odette Scharenborg and Diyuan Liu and Bao-Cai Yin and Jia Pan and Jianqing Gao and Cong Liu},
booktitle = {ICASSP 2022},
year = {2022}
}