SCAM-P: Spatial Channel Attention Module for Panoptic Driving Perception

Abstract

A high-precision, high-efficiency, and lightweight panoptic driving perception system is an essential part of autonomous driving for optimal maneuver planning of the autonomous vehicle. We propose a simple, lightweight, and efficient SCAM-P multi-task learning network that accomplishes three crucial tasks simultaneously for panoptic driving: vehicle detection, drivable area segmentation, and lane segmentation. To increase the representation power of the shared backbone of our multi-task network, we designed a novel SCAM module with spatially localized channel attention and channel localized spatial attention blocks. SCAM is a lightweight module that can be plugged into any CNN architecture to enhance the semantic features with negligible computational overhead. We integrate our SCAM module and design the SCAM-P network, which has a shared backbone for feature extraction and three independent heads to handle three tasks at the same time. We also designed a nano variant of our SCAM-P network to make it deployment-friendly on edge devices. Our SCAM-P network obtains competitive results on the BDD100K dataset with 81.1 % mAP50 for object detection, 91.6 % mIoU for drivable area segmentation, and 28.8 % IoU for lane segmentation. Our model is robust in various adverse weather conditions, such as rainy, snowy, and at night. Our SCAM-P network not only achieves improved performance but also runs efficiently in real-time at 230.5 FPS on the RTX 4090 GPU and 112.1 FPS on the Jetson Orin edge device.