Map-Free Visual Relocalization Enhanced by Instance Knowledge and Depth Knowledge

Abstract

Map-free visual relocalization computes camera pose using only a query image and a reference image. Therefore, it is hindered by challenges in feature-point matching and the absence of scale information in monocular images. These issues may cause significant rotational and metric errors, leading to localization failures. To address these challenges, we propose a map-free visual relocalization method enhanced with instance knowledge and depth knowledge. By utilizing instance-based matching, our approach improves the robustness of feature-point matching by focusing on relevant regions across scenes. Additionally, our depth estimation techniques provide reliable depth knowledge from a single image, improving scale recovery and reducing translation errors. Our method surpasses the previous state-of-the-art by 1.071m and 13.593° on the translation error and rotation error, respectively. Furthermore, we are also one of the winners in the Map-free Workshop & Challenge (ECCV2024), underscoring its superiority compared to concurrent approaches.