NeRF-VINS: A Real-time Neural Radiance Field Map-based Visual-Inertial Navigation System
Saimouli Katragadda, Woosik Lee, Yuxiang Peng, Patrick Geneva, Chuchu Chen, Chao Guo, Mingyang Li, Guoquan Huang
Abstract
Achieving efficient and consistent localization with a prior map remains challenging in robotics. Conventional keyframe-based approaches often suffer from sub-optimal viewpoints due to limited field of view (FOV) and/or constrained motion, thus degrading the localization performance. To address this issue, we design a real-time tightly-coupled Neural Radiance Fields (NeRF)-aided visual-inertial navigation system (VINS). In particular, by effectively leveraging the NeRF’s potential to synthesize novel views, the proposed NeRF-VINS overcomes the limitations of traditional keyframe-based maps (with limited views) and optimally fuses IMU, monocular images, and synthetically rendered images within an efficient filter-based framework. This tightly-coupled fusion enables efficient 3D motion tracking with bounded errors. We extensively validate the proposed NeRF-VINS against the state-of-the-art methods that use prior map information, and demonstrate its ability to perform real-time localization, at 15 Hz, on a resource-constrained Jetson AGX Orin embedded platform.
BibTeX
@inproceedings{icra2024_nerfvinsarealtim,
title = {NeRF-VINS: A Real-time Neural Radiance Field Map-based Visual-Inertial Navigation System},
author = {Saimouli Katragadda and Woosik Lee and Yuxiang Peng and Patrick Geneva and Chuchu Chen and Chao Guo and Mingyang Li and Guoquan Huang},
booktitle = {ICRA 2024},
year = {2024}
}