D3C2-Net: Dual-Domain Deep Convolutional Coding Network for Compressive Sensing

TL;DR

This paper develops a dual-domain optimization framework that combines the priors of image- and convolutional-coding-domains and offers generality to CS and other inverse imaging tasks and presents a Dual-Domain Deep Convolutional Coding Network (D3C2-Net), which enjoys the ability to efficiently transmit high-capacity self-adaptive convolutional features across all its unfolded stages.

Abstract

By mapping iterative optimization algorithms into neural networks (NNs), deep unfolding networks (DUNs) exhibit well-defined and interpretable structures and achieve remarkable success in the field of compressive sensing (CS). However, most existing DUNs solely rely on the image-domain unfolding, which restricts the information transmission capacity and reconstruction flexibility, leading to their loss of image details and unsatisfactory performance. To overcome these limitations, this paper develops a dual-domain optimization framework that combines the priors of (1) image- and (2) convolutional-coding-domains and offers generality to CS and other inverse imaging tasks. By converting this optimization framework into deep NN structures, we present a Dual-Domain Deep Convolutional Coding Network (D3C2-Net), which enjoys the ability to efficiently transmit high-capacity self-adaptive convolutional features across all its unfolded stages. Our theoretical analyses and experiments on simulated and real captured data, covering 2D and 3D natural, medical, and scientific signals, demonstrate the effectiveness, practicality, superior performance, and generalization ability of our method over other competing approaches and its significant potential in achieving a balance among accuracy, complexity, and interpretability. Code is available at https://github.com/lwq20020127/D3C2-Net.

Figures (11)

• Figure 1: Illustration of the classic concept of convolutional coding. An image $\mathbf{x}$ can be expressed by the summation of multiple image-level convolution results, i.e., there is $\mathbf{x}=\sum_{i=1}^{C}\left(\pmb{d}_i \ast \pmb{\alpha}_i\right) \in \mathbb{R}^{H\times W}$, where $\pmb{d}_i\in\mathbb{R}^{k\times k}$ represents the $i$-th dictionary filter, $\pmb{\alpha}_i\in\mathbb{R}^{H\times W}$ is the $i$-th coefficient map, $\ast$ denotes the convolution operator, and $C$ is the number of feature channels. The darker red (or blue) colors in visualized filter $\pmb{d}_i$ correspond to the positive (or negative) filter elements with larger absolute values. Compared to single-channel images, this feature-level representation naturally enjoys higher capacity.
• Figure 2: Illustration of the core concepts underlying the design of our introduced dual-domain deep unfolding networks (DUNs). (a) showcases the architecture of image-domain (ID)-based DUN, while (b) depicts the architecture of convolutional-coding-domain (CCD)-based DUN. In comparison to (a), which sequentially reconstructs the target image ($\{\mathbf{x}^{(t)}\}$), CCD-based DUN in (b) transmits high-dimensional features ($\{\pmb{\alpha}^{(t)}\}$) across all stages. Furthermore, (c) provides a conceptual visualization of the assumed convergence trajectories under single-domain and dual-domain constraints, demonstrating that the latter has the potential of achieving more accurate recoveries by leveraging the combined knowledge acquired in both of the two domains.
• Figure 3: Illustration of the comprehensive architecture of our $\text{D}^{\text{3}} \text{C}^{\text{2}}$-Net, which encompasses $T$ stages, unfolded from our dual-domain decoupled Algo. \ref{['alg:optimization']}. Each stage comprises an image-domain block (IDB) and a convolutional-coding-domain block (CCDB). Within the framework, $\mathbf{x}$ represents the fully-sampled image, $\mathbf{y}$ denotes the under-sampled measurement, and $\hat{\mathbf{x}}$ signifies the final output of $\text{D}^{\text{3}} \text{C}^{\text{2}}$-Net. The convolutional dictionary and coefficients are denoted by $\mathbf{D}$ and $\pmb{\alpha}$ respectively. Here, $k$ denotes the filter size of $\mathbf{D}$, $H$ and $W$ refer to the height and width of $\mathbf{x}$ and $\pmb{\alpha}$, and $C$ represents the number of channels.
• Figure 4: Illustration of the structural design of our unfolded $\text{D}^{\text{3}} \text{C}^{\text{2}}$-Net stage and its constituent components. In (a), we present the configuration of the $t$-th stage within $\text{D}^{\text{3}} \text{C}^{\text{2}}$-Net. An image-domain block (IDB) comprises a gradient descent module (GDM) and a proximal mapping network (PMN). Similarly, a convolutional-coding-domain block (CCDB) incorporates a data-term solving module (DTSM) and a prior-term solving network (PTSN). The specifics of the analytic GDM and DTSM are illustrated by Eqs. (\ref{['eq:GDM_impl']}) and (\ref{['eq:DTSM_impl']}) respectively. In (b), we provide the architectural design of the four sub-networks.
• Figure 5: Ablation studies on (a) dictionary filter size $k$, (b) feature number $C$, (c) unfolded stage number $T$. All experiments are performed on Set11 kulkarni2016reconnet with CS ratio $\gamma=30\%$.