Robustness of Panoptic Segmentation for Degraded Automotive Cameras Data (I)

Abstract

Abstract— Precise situational awareness is vital for the safe deployment of artificial intelligence in real-world scenarios, especially in assisted and automated driving (AAD) systems. Panoptic segmentation, which unifies semantic and instance segmentation, plays a key role in identifying objects, hazards, and drivable areas at the pixel level. However, the relationship between segmentation robustness and camera image quality remains insufficiently explored. To address this, we propose a unified pipeline to evaluate the robustness of panoptic segmentation models for automotive cameras, correlating their performance with eight traditional image quality assessment (IQA) metrics. We introduce D-Cityscapes+, a degraded dataset featuring 19 realistic automotive degradations at multiple severity levels, including novel darkness and snowfall models with veiling effects. Evaluation across 14 state-of-the-art backbones reveals: (1) large-particle degradations (e.g., lens droplets, heavy snow) cause severe performance drops and edge-concentrated errors; (2) Transformer-based models are more robust but limited by <2 FPS and >500 GFLOPs; (3) frequency-domain IQA metrics such as CW-SSIM strongly correlate with segmentation performance; and (4) generic image restoration does not consistently improve perception. The findings and benchmark (https://github.com/Warwick-Jocelyn/BRPS ) provide practitioners with diagnostic tools, datasets, and guidelines for developing robust, real-time panoptic