APKGC: Noise-enhanced Multi-Modal Knowledge Graph Completion with Attention Penalty
Yue Jian, Xiangyu Luo, Zhifei Li, Miao Zhang, Yan Zhang, Kui Xiao, Xiaoju Hou
Abstract
Multimodal knowledge graphs (MMKG) store structured world knowledge enriched with multimodal descriptive information. However, MMKG often faces the challenge of incompleteness. The primary objective of multimodal knowledge graph completion (MMKGC) is to predict missing entities within MMKG. Current MMKGC methods struggle with addressing the issue of over-trust attention and how to enhance the robustness of the model. To overcome these problems, we introduce APKGC, a noise-enhanced multimodal method for knowledge graph completion with attention penalty. APKGC effectively adjusts the attention scores in the language model and alleviates over-trust attention through a specifically designed attention penalty module. Additionally, an adaptive noise sampling module is proposed to supplement the entity's multimodal information, thereby enhancing the model's robustness. Experimental evaluation demonstrates that APKGC excels in overcoming these challenges. Compared to the existing state-of-the-art MMKGC model, APKGC improves Hit@1 by 3.3% on the DB15K dataset and by 3.4% on the MKG-W dataset.
BibTeX
@article{Jian_Luo_Li_Zhang_Zhang_Xiao_Hou_2025, title={APKGC: Noise-enhanced Multi-Modal Knowledge Graph Completion with Attention Penalty}, volume={39}, url={https://ojs.aaai.org/index.php/AAAI/article/view/33645}, DOI={10.1609/aaai.v39i14.33645}, abstractNote={Multimodal knowledge graphs (MMKG) store structured world knowledge enriched with multimodal descriptive information. However, MMKG often faces the challenge of incompleteness. The primary objective of multimodal knowledge graph completion (MMKGC) is to predict missing entities within MMKG. Current MMKGC methods struggle with addressing the issue of over-trust attention and how to enhance the robustness of the model. To overcome these problems, we introduce APKGC, a noise-enhanced multimodal method for knowledge graph completion with attention penalty. APKGC effectively adjusts the attention scores in the language model and alleviates over-trust attention through a specifically designed attention penalty module. Additionally, an adaptive noise sampling module is proposed to supplement the entity’s multimodal information, thereby enhancing the model’s robustness. Experimental evaluation demonstrates that APKGC excels in overcoming these challenges. Compared to the existing state-of-the-art MMKGC model, APKGC improves Hit@1 by 3.3% on the DB15K dataset and by 3.4% on the MKG-W dataset.}, number={14}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Jian, Yue and Luo, Xiangyu and Li, Zhifei and Zhang, Miao and Zhang, Yan and Xiao, Kui and Hou, Xiaoju}, year={2025}, month={Apr.}, pages={15005-15013} }