MultiCMET: A Novel Chinese Benchmark for Understanding Multimodal Metaphor

Dongyu Zhang; Jingwei Yu; Senyuan Jin; Liang Yang; Hongfei Lin

EMNLP 2023long findings0 citations

MultiCMET: A Novel Chinese Benchmark for Understanding Multimodal Metaphor

Dongyu Zhang, Jingwei Yu, Senyuan Jin, Liang Yang, Hongfei Lin

Abstract

Metaphor is a pervasive aspect of human communication, and its presence in multimodal forms has become more prominent with the progress of mass media. However, there is limited research on multimodal metaphor resources beyond the English language. Furthermore, the existing work in natural language processing does not address the exploration of categorizing the source and target domains in metaphors. This omission is significant considering the extensive research conducted in the fields of cognitive linguistics, which emphasizes that a profound understanding of metaphor relies on recognizing the differences and similarities between domain categories. We, therefore, introduce MultiCMET, a multimodal Chinese metaphor dataset, consisting of 13,820 text-image pairs of advertisements with manual annotations of the occurrence of metaphors, domain categories, and sentiments metaphors convey. We also constructed a domain lexicon that encompasses categorizations of metaphorical source domains and target domains and propose a Cascading Domain Knowledge Integration (CDKI) benchmark to detect metaphors by introducing domain-specific lexical features. Experimental results demonstrate the effectiveness of CDKI. The dataset and code are publicly available.

Multimodal metaphorBenchmarkChinese languageDomain lexiconMetaphor understanding

BibTeX

@inproceedings{
zhang2023multicmet,
title={Multi{CMET}: A Novel Chinese Benchmark for Understanding Multimodal Metaphor},
author={Dongyu Zhang and Jingwei Yu and Senyuan Jin and Liang Yang and Hongfei Lin},
booktitle={The 2023 Conference on Empirical Methods in Natural Language Processing},
year={2023},
url={https://openreview.net/forum?id=dnQI76LKQy}
}