Uncertainty-Aware Vision-Based Risk Object Identification Via Conformal Risk Tube Prediction

Abstract

We study object importance-based vision risk object identification (Vision-ROI), a key capability for intelligent driving systems. Existing approaches are deterministic and ignore uncertainty, potentially compromising safety. For example, fixed decision thresholds in ambiguous scenarios can cause premature or delayed risk detection and temporally unstable predictions. These issues worsen under diverse contexts with multiple interacting risks that perturb where and when risks occur. However, current vision methods lack a principled way to model uncertainty jointly across space and time, limiting adaptability to scene complexity. We propose Risk Tube Prediction, a unified formulation for modeling spatiotemporal risk uncertainty. We further introduce a conformal prediction framework to provide coverage guarantees for the true risks and yield calibrated risk scores and uncertainty estimates. Specifically, we employ risk-category–aware calibrators that consider distinct characteristics to reduce confused calibration. To evaluate, we present a new dataset and metrics probing diverse scenario configurations with multi-risk coupling effects. We systematically analyze factors affecting uncertainty estimation, including scenario variations, per-risk category behavior, and perception error propagation. Our method delivers substantial improvements over prior approaches, enhancing vision-ROI robustness and downstream performance, such as reducing nuisance braking alerts. For more qualitative results, please visit our project webpage: https://hcis-lab.github.io/CRTP/