GS-Loc: A Vision Foundation Model-Driven 3D Gaussian Splatting Framework for Robust Visual Relocalization

Abstract

Robust and accurate localization technologies are crucial for autonomous vehicles and mobile robots. Precisely perceiving the 3D environment and its semantic attributes in the real world greatly enhances the ability of these systems to perform localization tasks effectively. This paper proposes GS-Loc, the first visual relocalization framework based on a vision foundation model that utilizes 3D Gaussian Splatting (3DGS) as a map representation. GS-Loc leverages the powerful visual feature extraction capabilities of a vision foundation model as a Global Locator, performing global descriptor matching between query images and a reference image database. This enables each query image to be matched with the most optimal candidate image in the map and an initial pose within the map. Leveraging the high-dimensional feature embeddings extracted by vision foundation models, we propose the Foundation Feature-Consistent Matcher (FFC-Matcher), which establishes a feature similarity filtering mechanism to extract geometrically consistent feature subsets from high-quality images and depth maps rendered by the 3DGS model, thereby enabling pose refinement through 2D-3D feature matching. To enhance the efficiency of GS-Loc, we introduce sparsification of the 3DGS model by retaining only the Gaussians that contain relevant feature information. Extensive evaluations on the KITTI360 and ROBOTCAR datasets show that even with a model size reduced by approximately 50%, our approach still achieves state-of-the-art performance compared to other methods.