Some Intriguing Observations on the Learnt Matrices in Deep Unfolded Networks

Abstract

Deep-unfolded networks (DUNs) have set new performance benchmarks in fields such as compressed sensing, image restoration, and wireless communications. DUNs are built from conventional iterative algorithms, where an iteration is transformed into a layer/block of a network with learnable parameters. Despite their huge success, the reasons behind their superior performance over their iterative counterparts are not fully understood. This paper focuses on enhancing the explainability of DUNs by investigating potential reasons behind their superior performance over traditional iterative methods. We concentrate on the Learnt Iterative Shrinkage-Thresholding Algorithm (LISTA), a foundational contribution that achieves sparse recovery with significantly fewer layers than its iterative counterpart, ISTA. Our findings reveal that the learnt matrices in LISTA always have Gaussian distributed entries regardless of whether the sensing matrix is random Gaussian, Bernoulli, exponential, or uniform. The findings also show that the singular values of the learnt matrices exceed unity, despite which, the reconstruction scheme is stable. We conjecture that the activation function may have a role to play in ensuring stability. We also present an unbiasing technique that substantially improves the sparse recovery performance by reestimating the amplitudes based on the converged support.