Multi-Degradation Oriented Deep Unfolding Model for Hyperspectral Image Reconstruction

Abstract

Deep unfolding framework effectively combines model-driven and data-driven approaches, which generally contains a data reconstruction error term and a prior learning network, and has made significant progresses in hyperspectral image (HSI) reconstruction. However, existing methods face challenges in generalization and representation for high-dimensional HSI data, which are evident in two main areas: 1) the reliance on a fixed sensing mask limits the ability to generalize when reconstructing compressive measurements that are out of distribution, and 2) the prior representation network struggles to accurately represent high-dimensional data in both spatial and spectral domains. To address the above challenges, this study exploits a novel deep unfolding model (DUM), named Multi-Degradation Oriented DUM (MDO-DUM), designed to enhance both inverse projection and prior learning. Our approach improves generalization by training the DUM with samples synthesized using a variety of masks and integrating a mask-aware data modeling module (MADM). The MADM module works in conjunction with both the data reconstruction term and the prior learning network to facilitate degradation-aware projection and context-aware representation. For robust prior representation, we employ a spatial-spectral transformer that models both non-local spatial and spectral dependencies, effectively capturing the 3D attributes of HSIs. Additionally, we incorporate feature interactions across stages to capture diverse contexts and use auxiliary losses at each stage to boost the recovery performance. Extensive experiments on both simulated and real-world datasets demonstrate that our method surpasses existing state-of-the-art HSI reconstruction techniques.