Vision, Deduction and Alignment: An Empirical Study on Multi-Modal Knowledge Graph Alignment
Yangning Li, Jiaoyan Chen, Yinghui Li, Yuejia Xiang, Xi Chen, Hai-Tao Zheng
Abstract
Entity alignment (EA) for knowledge graphs (KGs) plays a critical role in knowledge engineering. Existing EA methods mostly focus on utilizing the graph structures and entity attributes (including literals), but ignore images that are common in modern multi-modal KGs. In this study we first constructed Multi-OpenEA — eight large-scale, image-equipped EA benchmarks, and then evaluated some existing embedding-based methods for utilizing images. In view of the complementary nature of visual modal information and logical deduction, we further developed a new multi-modal EA method named LODEME using logical deduction and multi-modal KG embedding, with state-of-the-art performance achieved on Multi-OpenEA and other existing multi-modal EA benchmarks.
BibTeX
@inproceedings{icassp2023_visiondeductiona,
title = {Vision, Deduction and Alignment: An Empirical Study on Multi-Modal Knowledge Graph Alignment},
author = {Yangning Li and Jiaoyan Chen and Yinghui Li and Yuejia Xiang and Xi Chen and Hai-Tao Zheng},
booktitle = {ICASSP 2023},
year = {2023}
}