MAP: Supporting Multimodal Knowledge Graph Completion via Augmented Modality Alignment and Instance Preserving

Abstract

Multimodal knowledge graphs (KGs) have found widespread applications in data integration and processing, yet existing multimodal knowledge graphs are often highly incomplete, which impedes their wide adoption. Thereby multimodal knowledge graph completion (MKGC) has attracted widespread attention. However, the heterogeneity of multiple modalities degenerates the representations’ capacity to model modalityshared discriminative knowledge. The state-of-the-art approach addresses this challenge by aligning the modality distributions by adopting a Sinkhorn-based approach, but such an approach is computationally expensive and the practical sampling strategy largely degrades the model performance. Therefore we propose the augmented modality distribution alignment module, which imposes the generalized Radon transform-based approach to perform efficient and accurate distribution alignment. Yet the alignment may result in undesirable instance-level feature structure disorder. We thus propose the relation-aware instance preserving module. Empirical comparisons on well-established MKGC benchmarks demonstrate the effectiveness of proposed method.