More than Text: Multi-modal Chinese Word Segmentation
Dong Zhang, Zheng Hu, Shoushan Li, Hanqian Wu, Qiaoming Zhu, Guodong Zhou
Abstract
Chinese word segmentation (CWS) is undoubtedly an important basic task in natural language processing. Previous works only focus on the textual modality, but there are often audio and video utterances (such as news broadcast and face-to-face dialogues), where textual, acoustic and visual modalities normally exist. To this end, we attempt to combine the multi-modality (mainly the converted text and actual voice information) to perform CWS. In this paper, we annotate a new dataset for CWS containing text and audio. Moreover, we propose a time-dependent multi-modal interactive model based on Transformer framework to integrate multi-modal information for word sequence labeling. The experimental results on three different training sets show the effectiveness of our approach with fusing text and audio.
BibTeX
@inproceedings{zhang-etal-2021-text,
title = "More than Text: Multi-modal {C}hinese Word Segmentation",
author = "Zhang, Dong and
Hu, Zheng and
Li, Shoushan and
Wu, Hanqian and
Zhu, Qiaoming and
Zhou, Guodong",
editor = "Zong, Chengqing and
Xia, Fei and
Li, Wenjie and
Navigli, Roberto",
booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)",
month = aug,
year = "2021",
address = "Online",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2021.acl-short.70/",
doi = "10.18653/v1/2021.acl-short.70",
pages = "550--557"
}