CoDifFu: Diffusion-Based Collaborative Perception with Efficient Heterogeneous Feature Fusion

Abstract

Multi-Agent collaborative perception is currently experiencing a surge in attention as a novel approach to addressing autonomous driving challenges. Despite advances in previous efforts, challenges remain due to various dilemmas in the perception process, such as imperfect localization and collaboration heterogeneity. To tackle these issues, we propose CoDifFu, a novel diffusion-based collaborative perception framework that enhances robustness against localization uncertainty and improves efficiency in heterogeneous feature fusion. A diffusion-based detection head progressively denoises object centers through a learnable reverse process. During training, the center coordinates of objects diffuse from the ground truth to the Gaussian distribution, then the network learns to reverse the diffusion process. In the inference, the model progressively refines a set of random centers of boxes to align with the ground truth centers. Moreover, we devised a confidence-guided multi-agent communication module(CMC), utilizing the confidence map as guidance to effectively achieve complementary feature fusion of multi-agent’s features and alleviates collaboration heterogeneity. To thoroughly evaluate CoDifFu, we consider 3D object detection in both real-world and simulation scenarios. Extensive experiments demonstrate the superiority of CoDifFu and the effectiveness of all its vital components. The code will be released.