A Novel Split Deep Unfolding Transformer for Pan-Sharpening

Abstract

Pan-sharpening is a commonly employed strategy to obtain high-resolution multispectral (HRMS) images. Existing deep unfolding networks for pan-sharpening suffer from ineffectively establishing the relationship between panchromatic (PAN) images and generated noisy HRMS (GN-HRMS) images in PAN-guided image denoising, lacking the support of physical models. In this paper, we first design a degradation-fusion-aware unfolding framework (DF-UF) to separate the processing of PAN-prior in PAN-guided image denoising into an individual module, PAN-prior processor, for better integrating physical models. Then, we derive a flexible intensity-hue-saturation (F-IHS) to act as the PAN-prior processor, which models the relationship between PAN images and GN-HRMS images in terms of intensity components through the intensity-hue-saturation (IHS) theory. Finally, plugging F-IHS into DF-UF, we propose a degradation-intensity-aware unfolding transformer (DIUT) to address the problem of incomplete utilization of PAN images in the denoising process. Extensive experiments on diverse scenes show that the performance of DIUT surpasses existing state-of-the-art methods.