Efficient Hierarchical Stripe Attention for Lightweight Image Super-Resolution

Abstract

Following the successful integration of Transformers in computer vision, several Transformer-based approaches have emerged, surpassing the previously dominant CNN-based techniques. However, it is observed that conventional self-attention involves redundant operations that can be further optimized. In this context, we propose a novel framework for lightweight super-resolution, termed as Hierarchical Stripe-attention Super-Resolution Network (HSSRNet). Specifically, our approach involves the fusion of correlated patches in each layer to access global pixel information. Subsequently, we introduce a stripe intra-patch self-attention (stripe IPSA) mechanism to model long-range dependencies efficiently. To further reduce computational complexity, we approximate the original attention map using two low-rank matrices. Finally, we employ PixelShuffle in conjunction with convolutions for upsampling. Extensive experiments demonstrate the effectiveness of our proposed modules, and the results indicate that our method achieves commendable performance on four benchmark datasets.