CMGFA: A BEV Segmentation Model Based on Cross-Modal Group-Mix Attention Feature Aggregator

Abstract

Bird's eye view (BEV) segmentation map is a recent development in autonomous driving that provides effective environmental information, such as drivable areas and lane dividers. Most of the existing methods use cameras and LiDAR as inputs for segmentation and the fusion of different modalities is accomplished through either concatenation or addition operations, which fails to exploit fully the correlation and complementarity between modalities. This letter presents the CMGFA (Cross-Modal Group-mix attention Feature Aggregator), an end-to-end learning framework that can adapt to multiple modal feature combinations for BEV segmentation. The CMGFA comprises the following components: i) The camera has a dual-branch structure that strengthens the linkage between local and global features. ii) Multi-head deformable cross-attention is applied as cross-modal feature aggregators to aggregate camera, LiDAR, and Radar feature maps in BEV for implicit fusion. iii) The Group-Mix attention is used to enrich the attention map feature space and enhance the ability to segment between different categories. We evaluate our proposed method on the nuScenes and Argoverse2 datasets, where the CMGFA significantly outperforms the baseline.