OPRNet: Object-Centric Point Reconstruction Network for Multimodal 3D Object Detection in Adverse Weathers

Abstract

The development of a multimodal fusion technique utilizing LiDAR-camera data has enabled precise 3D object detection for self-driving vehicles, particularly in ideal conditions with clear weather. Nevertheless, adverse weathers such as fog, snow, and rain remain a challenge for existing multimodal methods. These conditions lead to a reduced density of point clouds as a result of laser signal occlusion and attenuation. Additionally, as the distance grows, the point cloud becomes sparser, further challenging object detection tasks. To address these problems, we introduce a point reconstruction network employing equirectangular projection tailored for multimodal 3D object detection. This network incorporates a range-constrained noise filter to remove noise caused by adverse weather and an object-centric point generator designed to flexibly generate points for distant objects. Moreover, we propose a dual 2D auxiliary module to enhance image features and support the point reconstruction. Experimental evaluations conducted on adverse weather datasets demonstrate that the suggested approach surpasses current techniques. The implementation can be accessed at https://github.com/jhyoon964/oprnet.