FourierPlace: A Vision-Language Localization Framework Based on Frequency Domain Representations

Abstract

Language-guided localization within 3D environments continues to pose a significant challenge for autonomous systems, primarily due to the need for precise alignment between sparse point cloud data and inherently ambiguous natural language descriptions. To address this, we present a novel vision-language localization framework, namely FourierPlace, that leverages frequency-domain representations to enhance the alignment of complex geometric features with imprecise linguistic cues. At the core of our approach is the Frequency Fusion Enhancement (FFE) module, which converts raw point cloud data into frequency-domain signals. Complementing this, the Fourier Gate Attention (FGA) mechanism operates on these frequency-domain features to strengthen cross-modal correspondence. Furthermore, we introduce the Hierarchical Language Understanding Network (HiLUNet), which progressively refines linguistic features through a multi-stage architecture. Additionally, the Multiscale Cascade Cross-Attention (MCCA) module incorporates geometric information at multiple scales in the fine stage. Experiments conducted on the KITTI360Pose benchmark demonstrate that FourierPlace achieves state-of-the-art performance, outperforming existing methods with 3.7% improvement in Top-1 coarse retrieval accuracy and 4.0% increase in Top-1 localization accuracy within a 15-meter threshold on the test set. Our proposed framework presents a robust and scalable solution for language-guided localization in large-scale autonomous applications, including delivery robotics and augmented reality (AR) navigation systems. Our code is now available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/nuozimiaowu/FourierPlace</uri>.