CAMDet: Condition-Adaptive Multispectral Object Detection Using a Visible-Thermal Translation Model

Abstract

In multispectral object detection, integrating visible and thermal features offers significant advantages, particularly for autonomous driving in low-light environments. However, collecting a large dataset of pixel-aligned visible and thermal image pairs is labor-intensive, and achieving real-time alignment in operational driving systems remains a challenge. Additionally, the representation of thermal images varies across different camera types, complicating generalization, while visible images are often prone to environmental noise. To address these issues, we present CAMDet, a novel network that selectively fuses visible and thermal images based on day/night conditions. To compensate for missing modality information, we incorporate a diffusion model-based Visible-Thermal conversion method to synthesize the missing modality. An attention-based Modality Feature Refinement (MFR) module further enhances feature quality by reducing uncertainties in the generated images. Comprehensive experiments on the FLIR and LLVIP datasets show that CAMDet significantly outperforms single-modality detection methods and surpasses multi-modality baseline models that depend on visible-thermal image pairs.