Dual Attention for Space-Time Video Super-Resolution

Abstract

Space-time video super-resolution (STVSR) aims to generate videos with higher resolution (HR) and higher frame rate (HFR). In this paper, we propose a dual-attention STVSR model (DAST) to combine local features with global dependencies. DAST is a compact one-stage STVSR method that tightly combines spatial video super-resolution (S-VSR) and temporal video super-resolution (T-VSR) tasks. Specifically, we design a dual-attention network (DANet). First, we design a high-frequency filter attention block (HFAB) to extract local spatial-temporal information from video frames and enhance the model’s ability to capture local details. Then, the global dependencies in the features are captured by multi-head self attention (MSA) to construct global context information. In addition, a nonlinear residual block (NLRB) is designed to enhance the expressiveness of the model and make the input features more adaptable to each layer of the network. Extensive experiments demonstrate that our method achieves better performance in the STVSR task.