LiDAR-BIND: Multi-Modal Sensor Fusion Through Shared Latent Embeddings

Abstract

This letter presents LiDAR-BIND, a novel sensor fusion framework aimed at enhancing the reliability and safety of autonomous vehicles (AVs) through a shared latent embedding space. With this method, the addition of different modalities, such as sonar and radar, into existing navigation setups becomes possible. These modalities offer robust performance even in challenging scenarios where optical sensors fail. Leveraging a shared latent representation space, LiDAR-BIND enables accurate modality prediction, allowing for the translation of one sensor's observations into another, thereby overcoming the limitations of depending solely on LiDAR for dense point-cloud generation. Through this, the framework facilitates the alignment of multiple sensor modalities without the need for large synchronized datasets across all sensors. We demonstrate its usability in SLAM applications, outperforming traditional LiDAR-based approaches under degraded optical conditions.