Spatially-variant Blur Degradation Model Based on Depth Estimation

Abstract

It is well known that the number of aligned images in the single image super-resolution (SISR) models training is limited. Synthesizing data is an effective way to address this issue. However, many degradation models only consider using spatially-invariant blur kernels to blur high-resolution (HR) images, which leads to a gap between synthetic and real-world low-resolution (LR) images. To this end, it is necessary to design a degradation model that can generate the authentic spatially-variant blur kernels. In this paper, we propose to integrate the degradation model with depth estimation to obtain the depth information of HR images, and use it to design a practical degradation model. Specifically, we first estimate the depth of HR images by a pretrained depth estimation network and use a convolution layer to extract the features of the depth images. Subsequently, we concatenate the feature map with the corresponding HR images, and put it through several convolution blocks to produce the estimated blur kernels. To further increase the covered degradation space, we also add predefined noise and JPEG compression to our designed degradation model. The spatially-variant blur degradation model (SVBDM) can predict pixel-wise blur kernels effectively and ensure SISR models can generate favorable results under different degradation. Extensive experiments are performed for various degradation methods and SISR backbones to demonstrate the effectiveness of the SVBDM.