Continual Multimodal Knowledge Graph Construction
Xiang Chen, Jingtian Zhang, Xiaohan Wang, Ningyu Zhang, Tongtong Wu, Yuxiang Wang, Yongheng Wang, Huajun Chen
Abstract
Current Multimodal Knowledge Graph Construction (MKGC) models struggle with the real-world dynamism of continuously emerging entities and relations, often succumbing to catastrophic forgetting—loss of previously acquired knowledge. This study introduces benchmarks aimed at fostering the development of the continual MKGC domain. We further introduce the MSPT framework, designed to surmount the shortcomings of existing MKGC approaches during multimedia data processing. MSPT harmonizes the retention of learned knowledge (stability) and the integration of new data (plasticity), outperforming current continual learning and multimodal methods. Our results confirm MSPT's superior performance in evolving knowledge environments, showcasing its capacity to navigate the balance between stability and plasticity.
BibTeX
@inproceedings{ijcai2024p688,
title = {Continual Multimodal Knowledge Graph Construction},
author = {Chen, Xiang and Zhang, Jingtian and Wang, Xiaohan and Zhang, Ningyu and Wu, Tongtong and Wang, Yuxiang and Wang, Yongheng and Chen, Huajun},
booktitle = {Proceedings of the Thirty-Third International Joint Conference on
Artificial Intelligence, {IJCAI-24}},
publisher = {International Joint Conferences on Artificial Intelligence Organization},
editor = {Kate Larson},
pages = {6225--6233},
year = {2024},
month = {8},
note = {Main Track},
doi = {10.24963/ijcai.2024/688},
url = {https://doi.org/10.24963/ijcai.2024/688},
}