PG-Match: A Pose-Guided Generalizable Framework for Semi-Dense Feature Matching

Abstract

Feature matching is a fundamental technique in visual perception, essential for tasks such as 3D reconstruction, SLAM, and visual localization. Existing detector-free methods often struggle with generalization due to their reliance on depth data, which is not available in many datasets. We propose PG-Match, a detector-free feature matching framework that leverages pose supervision instead of depth-based supervision, improving its generalization across diverse environments. Additionally, we introduce a Differentiable Outlier Rejection Module (DORM) to enhance global consistency and increase the inlier ratio. A coarse-to-fine matching strategy is employed for efficiency, where specially designed confidence scores are utilized to guide the sampling process. This ensures efficient convergence and avoids local optima. Experiments on the widely used MegaDepth-1500 dataset demonstrate that PG-Match consistently outperforms state-of-the-art approaches, highlighting the effectiveness of its pose-guided design. Additionally, experiments on the depth-free PhotoTourism dataset further evaluate generalization of PG-Match, and its performance is also assessed in a downstream Structure from Motion (SfM) task.