Spectral Low-Rank Attention with Flow-Based Refinement for Spectral Reconstruction

Abstract

Spectral super-resolution (SSR) from RGB images, which involves reconstructing hyperspectral images (HSIs) from color images, has recently received great attention. While convolutional neural network (CNN)-based methods have demonstrated strong performance, they often overlook the self-similarity across the spectral dimension of HSIs. Transformer-based approaches have addressed this limitation by leveraging self-attention mechanisms to capture spectral correlations. However, these methods encounter computational and memory overheads that scale quadratically with the size of the HSIs. To overcome these challenges, we introduce a novel Spectral-wise Low-Rank Attention (SLORA) mechanism that captures inter-spectral consistency in a low-dimensional space, thereby reducing both computational costs and model complexity. Additionally, we propose a flow-based refinement module to enhance generalization and performance on unseen HSIs. Experimental results from the NTIRE 2022 spectral reconstruction challenge and the spectral snapshot compression imaging task datasets validate the superiority of our method over state-of-the-art approaches.