Bat-VUFN: Bat-Inspired Visual-and-Ultrasound Fusion Network for Robust Perception in Adverse Conditions

Abstract

Environmental factors like weather and road conditions significantly impact object recognition in autonomous vehicles. While cameras provide rich semantic information, their reliance on electromagnetic waves makes them vulnerable to performance degradation in adverse conditions such as low light and rain. In contrast, ultrasonic sensors offer reliable short-range detection, unaffected by such conditions. We introduce Bat-VUFN, a bio-inspired multi-sensory system that merges camera and ultrasonic data using an Input Quality Score (IQS)-based fusion technique to enhance near-field perception in challenging environments. Bat-VUFN dynamically adjusts sensor contributions based on prevailing conditions, achieving impressive results on the K-Bat dataset (average precision: 0.95, MAE: 0.52m, RMSE: 0.55m), demonstrating its robustness in adverse scenarios.