VSG-SLAM:A Dense Visual Semantic SLAM with Gaussian Splatting

Abstract

Simultaneous Localization and Mapping (SLAM) is critical for real-time robotic applications, enabling precise localization and comprehensive scene reconstruction. Recent advances in 3D Gaussian Splatting (3DGS) enable high-quality view synthesis and rapid rendering, yet robust and consistent semantic scene representation remains under-explored. In this work, we introduce a dense semantic SLAM framework that integrates high-dimensional semantic features with an explicit 3D Gaussian-based scene representation to address these challenges. Our approach employs a lightweight projection layer that maps low-dimensional semantic features to high-dimensional embeddings, a coarse-to-fine and semantically informed camera tracking strategy that robustly estimates camera poses, a mapping module that incrementally refines the Gaussian map by simultaneously leveraging geometric and photometric cues alongside semantic information and a covisibility-based local bundle adjustment module for joint optimization of camera poses and Gaussian parameters. Extensive experiments on synthetic and real-world indoor datasets demonstrate that our framework achieves superior reconstruction quality, enhanced semantic segmentation accuracy, and competitive camera pose estimation.