Summary on the Multimodal Information Based Speech Processing (MISP) 2022 Challenge
Hang Chen, Shilong Wu, Yusheng Dai, Zhe Wang, Jun Du, Chin-Hui Lee, Jingdong Chen, Shinji Watanabe
Abstract
The Multimodal Information based Speech Processing (MISP) 2022 challenge aimed to enhance speech processing performance in harsh acoustic environments by leveraging additional modalities such as video or text. The challenge included two tracks: audio-visual speaker diarization (AVSD) and audio-visual diarization and recognition (AVDR). The training material was based on previous MISP 2021 recordings, but we have accurately synchronized audio and visual data. Additionally, a new evaluation set was provided. This paper gives an overview of the challenge setup, presents the results, and summarizes the effective techniques employed by the participants. We also analyze the current technical challenges and suggest directions for future research in AVSD and AVDR.
BibTeX
@inproceedings{icassp2023_summaryonthemult,
title = {Summary on the Multimodal Information Based Speech Processing (MISP) 2022 Challenge},
author = {Hang Chen and Shilong Wu and Yusheng Dai and Zhe Wang and Jun Du and Chin-Hui Lee and Jingdong Chen and Shinji Watanabe and Sabato Marco Siniscalchi and Odette Scharenborg and Diyuan Liu and Bao-Cai Yin and Jia Pan and Jianqing Gao and Cong Liu},
booktitle = {ICASSP 2023},
year = {2023}
}