4DRC-OC: Online Calibration of 4D Millimeter Wave Radar-Camera With Depth Map Assistance

Abstract

The online calibration of 4D millimeter-wave radar and camera is crucial for advancing perception and SLAM technologies in complex environments. It eliminates the reliance on manual labeling, offering real-time and convenience. However, the sparse nature of 4D radar point clouds presents challenges in establishing correspondences with camera images. This paper proposes an online 4D radar-camera online calibration method (4DRC-OC) that utilizes unified depth map representations for auxiliary training, ensuring feature alignment and modal unification between the two sensors. Due to the limited useful information within sparse depth maps, 4DRC-OC uses dynamic convolution to adaptively capture detailed features. Furthermore, this paper designs a correlation module based on channel-wise fusion (CMCF) that computes correlations between error depth maps and RGB-derived depth maps, thereby enhancing features to facilitate extrinsic parameter regression. Experimental results on the Dual-Radar dataset validate the superiority of the proposed approach in extrinsic calibration.