AnytimeFusion: Parameter-free RGB Camera-Radar Sensor Fusion Algorithm in Complex Maritime Situations

Abstract

Determining the position of obstacles is crucial for unmanned vehicles, and, to achieve this, cameras and radar sensors are widely utilized. However, establishing correlation between two or more sensors proves challenging in the dynamically changing maritime environment. To solve these issues, we propose the AnytimeFusion algorithm. The key innovation of AnytimeFusion lies in the utilization of a parameter-free method that does not require accurate sensor alignment and calibration. The algorithm consists of the following four stages. First, calibration targets are selected in the maritime environment based on segmentation images. Second, radar and camera data are pre-fused to model the correlation of azimuth information. After completing the auto-calibration stages, Inverse Perspective Mapping (IPM) is employed to integrate the coordinate systems of the two sensors. To determine the parameters for this integration, optimization based on the Particle Swarm Optimization (PSO) method is employed. Finally, an Error Polygon for the positions of the camera and radar is generated, and sensor fusion is carried out based on this information. We validated our method through experiments conducted on real ships in complex maritime environments, achieving an average accuracy of 95.7%.