Cross-Modality Fusion Mamba for All-in-One Extreme Weather-Degraded Image Restoration

Abstract

A major obstacle for high-level tasks is the unpredictable image degradation. While several architectures proposed to address this, they fail under extreme degradation. Therefore, we introduce a novel cross-modality pipeline called AIRMamba, designed to holistically and robustly restore images degraded due to extreme weather. Specifically, we devise a strategy that utilizes infrared images to create compact high-frequency priors for the restoration process. Meanwhile, we leverage long-range modeling capability of Mamba to achieve both feature extraction and interaction. We emphasize extracting low-frequency representations from the ground truth and achieving this task through a regression-based approach. Consequently, AIRMamba can achieve reliable restoration through large-gap cross-domain guidance. To facilitate this task, we have constructed a cross-modality restoration benchmark, named WeatherInfrared. Our pipeline is simple, robust, and outperforms several state-of-the-art methods in benchmark evaluations.