LiDAR-VGGT: Cross-Modal Coarse-to-Fine Fusion for Globally Consistent and Metric-Scale Dense Mapping

Abstract

Reconstructing large-scale RGB point clouds is an important task in robotics, supporting perception, navigation, and scene understanding. Despite advances in LiDAR inertial visual odometry (LIVO), its performance remains highly sensitive to extrinsic calibration. Meanwhile, 3D vision foundation models, such as VGGT, suffer from limited scalability in large environments and inherently lack metric scale. To overcome these limitations, we propose LiDAR-VGGT, a novel framework that couples LiDAR inertial odometry with the state-of-the-art VGGT model through a two-stage coarse-to-fine fusion pipeline: First, a pre-fusion module robustly estimates per-session VGGT poses and RGB point clouds with a coarse metric scale. Then, a post-fusion module enhances cross-modal 3D similarity transformation, using bounding-box–based regularization to reduce scale distortions caused by inconsistent FOVs between LiDAR and camera sensors. Extensive experiments across multiple datasets demonstrate that LiDAR-VGGT achieves improved visual quality and density compared to VGGT-based methods and LIVO baselines. The implementation of our proposed novel RGB point cloud evaluation toolkit will be released as open source.