Inter-Scale Sure-Let Denoise with Structured Deep Image Prior: Interpretable Self-Supervised Learning

Abstract

This work proposes a novel image restoration technique inspired by the Ulyanov’s deep image prior (DIP) method. DIP uses a deep convolutional network as an image prior to generate a restored image from a random input one, which brings an advantage of no requirement of training data. However, one problem arises that the interpretability is low and it is not trivial to explain the need for the random input. This paper contributes to train a network by using the Stein’s unbiased risk estimator (SURE) with Monte-Carlo computation for self-supervised learning of image restoration. The Monte-Carlo calculation accepts a random input for computing divergence of the network, and allows us to interpret the reason why a random input is needed. Our framework trains an image restorer instead of an image generator, which simplifies the interpretation of the network structure. Thanks to this, the Luisier’s interscale linear expansion of the thresholding (LET) can become introduced to exploit the cross-correlation among extracted features. In order to avoid using group delay compensation and reduce the number of design parameters, a structured DIP is constructed by using non-separable oversampled lapped transform (NSOLT) instead of U-Net. By showing some results of denoising simulation, the significance of the proposed method is verified.