Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning of Patch-Based Smooth-Plus-Sparse Models for Image Reconstruction

Stanislas Ducotterd, Sebastian Neumayer, Michael Unser

TL;DR

This work tackles inverse imaging problems by learning a patch-based smooth-plus-sparse model that combines a data-consistent term with a per-patch dictionary-based regularizer. The approach casts reconstruction as a bilevel optimization: an inner solver computes the image $\mathbf{x}^*$ given dictionaries $\mathbf{D}$ and regularizer $R$, while an outer loop optimizes these parameters (and an analysis dictionary $\mathbf{Q}$) using implicit differentiation, yielding a two-layer convolutional network fixed point. A key design is the separation of low-frequency content via $\hat{\mathbf{P}}_k=(\mathbf{I}-\mathbf{Q}\mathbf{Q}^T)\mathbf{P}_k$, enabling a smooth component in the $\mathbf{Q}$-subspace and a sparse component from $\mathbf{D}$. The paper demonstrates gains over classical priors (TV, KSVD, BM3D) and competitive or superior performance to deep learning methods in denoising, super-resolution, and CS-MRI, with particular robustness when data are scarce or highly undersampled. The method provides interpretable decompositions and carries convergence guarantees due to its fixed-point CNN formulation, offering a principled alternative to large DL models in medical and scientific imaging.

Abstract

We aim at the solution of inverse problems in imaging, by combining a penalized sparse representation of image patches with an unconstrained smooth one. This allows for a straightforward interpretation of the reconstruction. We formulate the optimization as a bilevel problem. The inner problem deploys classical algorithms while the outer problem optimizes the dictionary and the regularizer parameters through supervised learning. The process is carried out via implicit differentiation and gradient-based optimization. We evaluate our method for denoising, super-resolution, and compressed-sensing magnetic-resonance imaging. We compare it to other classical models as well as deep-learning-based methods and show that it always outperforms the former and also the latter in some instances.

Learning of Patch-Based Smooth-Plus-Sparse Models for Image Reconstruction

TL;DR

This work tackles inverse imaging problems by learning a patch-based smooth-plus-sparse model that combines a data-consistent term with a per-patch dictionary-based regularizer. The approach casts reconstruction as a bilevel optimization: an inner solver computes the image given dictionaries and regularizer , while an outer loop optimizes these parameters (and an analysis dictionary ) using implicit differentiation, yielding a two-layer convolutional network fixed point. A key design is the separation of low-frequency content via , enabling a smooth component in the -subspace and a sparse component from . The paper demonstrates gains over classical priors (TV, KSVD, BM3D) and competitive or superior performance to deep learning methods in denoising, super-resolution, and CS-MRI, with particular robustness when data are scarce or highly undersampled. The method provides interpretable decompositions and carries convergence guarantees due to its fixed-point CNN formulation, offering a principled alternative to large DL models in medical and scientific imaging.

Abstract

We aim at the solution of inverse problems in imaging, by combining a penalized sparse representation of image patches with an unconstrained smooth one. This allows for a straightforward interpretation of the reconstruction. We formulate the optimization as a bilevel problem. The inner problem deploys classical algorithms while the outer problem optimizes the dictionary and the regularizer parameters through supervised learning. The process is carried out via implicit differentiation and gradient-based optimization. We evaluate our method for denoising, super-resolution, and compressed-sensing magnetic-resonance imaging. We compare it to other classical models as well as deep-learning-based methods and show that it always outperforms the former and also the latter in some instances.

Paper Structure

This paper contains 21 sections, 1 theorem, 14 equations, 13 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contribution
  3. Related Work
  4. Dictionary Learning
  5. Convolutional Dictionaries
  6. Variational Image Decomposition
  7. Method
  8. Training of the Model---Inner Optimization
  9. Training of the Model---Outer Optimization
  10. Dictionaries
  11. Regularizers
  12. Experiments
  13. Training on Denoising
  14. Visualization of the Model
  15. Inverse Problems
  16. ...and 6 more sections

Key Result

Lemma 1

The operator $\sum_{k=1}^n \hat{\mathbf{P}}_k^T \hat{\mathbf{P}}_k$ is Linear Shift-Invariant.

Figures (13)

  • Figure 1: The proximal operator of the non-convex model and the corresponding potential computed numerically when $\gamma = 2$ and $\tau = 1$.
  • Figure 2: Learned atoms for the NCPR model with $\sigma=25/255$.
  • Figure 3: Denoising with $\sigma=25/255$: decomposition of $\mathbf{x}^*$ into the free-atom and constrained-atom dictionaries. The last column shows the cost associated with each local patch $\mathbf{P}_k \mathbf{x}^*$.
  • Figure 4: Reconstruction performances for the super-resolution task.
  • Figure 5: Reconstruction performances for 8-fold subsampling and PD data.
  • ...and 8 more figures

Theorems & Definitions (2)

  • Lemma 1
  • proof