DGJA: Dependency Graph-enhanced Joint Attention Structure for Multimodal Sarcasm Detection

Abstract

Multimodal sarcasm detection (MSD) leverages multimodal data, including both images and text, to detect whether the input content contains sarcastic information. Despite recent advances, existing MSD approaches often overlook the imbalance in sarcastic content between text and image modalities, where text typically carries more sarcastic cues conveyed through complex semantic relationships. To address this issue, we propose a dependency graph-enhanced joint attention network that integrates the flat representations extracted from the joint attention mechanisms and the graph-based representations learned from the dependency graph. Specifically, we design a joint attention module to capture the rich sarcastic cues and semantic relationships within the text, and use the graph-based representations to enhance the flat representations. Evaluated on the HFM dataset, our method achieved 0.54% improvement in F1 score and 0.81% in accuracy, demonstrating its effectiveness.