Deep Regularized Compound Gaussian Network for Solving Linear Inverse Problems

TL;DR

Two novel approaches for linear inverse problems that permit problem-specific statistical prior selections within the compound Gaussian (CG) class of distributions are devised, one of which is a novel deep regularized (DR) neural network that learns the prior information.

Abstract

Incorporating prior information into inverse problems, e.g. via maximum-a-posteriori estimation, is an important technique for facilitating robust inverse problem solutions. In this paper, we devise two novel approaches for linear inverse problems that permit problem-specific statistical prior selections within the compound Gaussian (CG) class of distributions. The CG class subsumes many commonly used priors in signal and image reconstruction methods including those of sparsity-based approaches. The first method developed is an iterative algorithm, called generalized compound Gaussian least squares (G-CG-LS), that minimizes a regularized least squares objective function where the regularization enforces a CG prior. G-CG-LS is then unrolled, or unfolded, to furnish our second method, which is a novel deep regularized (DR) neural network, called DR-CG-Net, that learns the prior information. A detailed computational theory on convergence properties of G-CG-LS and thorough numerical experiments for DR-CG-Net are provided. Due to the comprehensive nature of the CG prior, these experiments show that DR-CG-Net outperforms competitive prior art methods in tomographic imaging and compressive sensing, especially in challenging low-training scenarios.

Figures (7)

• Figure 1: End-to-end network structure for DR-CG-Net, the unrolled deep neural network of Algorithm \ref{['main:alg:CG-LS']}, is shown in (\ref{['main:fig:DR-CG-Net']}). DR-CG-Net consists of an input block, $L_0$, initialization block, $Z_0$, $K+1$ Tikhonov blocks, $U_k$, $K$ complete scale variable mappings, $\mathcal{Z}_k$, a Hadamard product block, $C$, and an optional refinement block, $\mathcal{G}$. Each $\mathcal{Z}_k$, with structure in (\ref{['main:fig:DR-CG-Net scale mapping module']}), consists of $J$ updates $g_k^{(j)}$ further detailed in (\ref{['main:fig:DR-CG-Net z update Module']}). Each $g_k^{(k)}$ consists of a data fidelity gradient descent step, $r_k^{(j)}$, added to a convolutional neural network, $\mathcal{W}_k^{(j)}$ in (\ref{['main:fig:DR-CG-Net CNN']}), and corresponds to an intermediate update of the $\bm{z}$ variable.
• Figure 2: Structure of the $\bm{u}$ update block, $U_k$, for using gradient descent steps with Nesterov momentum.
• Figure 3: Average test image reconstruction SSIM when varying the amount of CIFAR10 data in training nine machine learning-based image reconstruction methods. Here, the sensing matrices, $\Psi$, are a Radon transform at 15, 10, or 6 uniformly spaced angles, $\Phi = I$, and the measurement SNR is 60dB or 40dB. Our DR-CG-Net method outperforms the compared prior art methods, in all scenarios, and does so appreciably in low training.
• Figure 4: Image reconstructions (SSIM) using our DR-CG-Net and six competitive deep learning methods on a $128\times 128$ test scan after training with only 20 samples. The sensing matrix, $\Psi$, is a Radon transform at 60 uniformly spaced angles, $\Phi = I$, and each measurement has an SNR of 60dB. Our DR-CG-Net methods perform best visually and by SSIM.
• Figure 5: Test image reconstruction quality for deep learning-based image estimation methods trained on only 20 samples. In all cases, our method, DR-CG-Net given by the top solid red bar, outperforms the other approaches.

Theorems & Definitions (20)

• Proposition 1
• Theorem 2
• Proposition 3
• Corollary 4
• Proposition 5
• Definition 1
• Lemma 1
• proof
• Lemma 2
• proof