SG-Reg: Generalizable and Efficient Scene Graph Registration

Abstract

This paper addresses the challenge of registering two rigid semantic scene graphs, an essential capability for autonomous agents to align with remote agents or prior maps. Traditional methods rely on hand-crafted descriptors or ground-truth annotations, limiting their applicability in real-world scenarios. To address these issues, we propose a scene graph network that encodes multiple semantic node modalities: open-set semantic features, local topology with spatial awareness, and shape features. These modalities are fused to form compact semantic node representations for matching layers to perform coarse-to-fine correspondence search. A robust pose estimator in the back-end determines the transformation based on these correspondences. Our approach preserves a sparse, hierarchical scene representation, requiring fewer GPU resources and less communication bandwidth in multi-agent tasks. Additionally, we introduce a novel data generation method using vision foundation models and a semantic mapping module, avoiding the need for ground-truth annotations. We validate our method on a two-agent SLAM benchmark, demonstrating superior registration success and lower communication bandwidth.