Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multivariate Fields of Experts for Convergent Image Reconstruction

Stanislas Ducotterd, Michael Unser

TL;DR

The multivariate fields of experts, a new framework for the learning of image priors is introduced, which outperforms comparable univariate models and achieves performance close to that of deep-learning-based regularizers while being significantly faster, requiring fewer parameters, and being trained on substantially fewer data.

Abstract

We introduce the multivariate fields of experts, a new framework for the learning of image priors. Our model generalizes existing fields of experts methods by incorporating multivariate potential functions constructed via Moreau envelopes of the $\ell_\infty$-norm. We demonstrate the effectiveness of our proposal across a range of inverse problems that include image denoising, deblurring, compressed-sensing magnetic-resonance imaging, and computed tomography. The proposed approach outperforms comparable univariate models and achieves performance close to that of deep-learning-based regularizers while being significantly faster, requiring fewer parameters, and being trained on substantially fewer data. In addition, our model retains a high level of interpretability due to its structured design. It is supported by theoretical convergence guarantees which ensure reliability in sensitive reconstruction tasks.

Multivariate Fields of Experts for Convergent Image Reconstruction

TL;DR

The multivariate fields of experts, a new framework for the learning of image priors is introduced, which outperforms comparable univariate models and achieves performance close to that of deep-learning-based regularizers while being significantly faster, requiring fewer parameters, and being trained on substantially fewer data.

Abstract

We introduce the multivariate fields of experts, a new framework for the learning of image priors. Our model generalizes existing fields of experts methods by incorporating multivariate potential functions constructed via Moreau envelopes of the -norm. We demonstrate the effectiveness of our proposal across a range of inverse problems that include image denoising, deblurring, compressed-sensing magnetic-resonance imaging, and computed tomography. The proposed approach outperforms comparable univariate models and achieves performance close to that of deep-learning-based regularizers while being significantly faster, requiring fewer parameters, and being trained on substantially fewer data. In addition, our model retains a high level of interpretability due to its structured design. It is supported by theoretical convergence guarantees which ensure reliability in sensitive reconstruction tasks.

Paper Structure

This paper contains 28 sections, 2 theorems, 49 equations, 9 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Description of the Model
  4. Notations and Definitions
  5. Generalization of the WCRR
  6. General Setting
  7. Optimization
  8. Training on Denoising
  9. Bilevel Formulation
  10. Inner Optimization (Forward Pass)
  11. Outer Optimization (Backward Pass)
  12. Implementation Details
  13. Results and Ablation Studies
  14. Visualization and Analysis of the Learned Model
  15. Inverse Problems
  16. ...and 13 more sections

Key Result

Theorem 1

If $\mathbf Q_k$ is such that $\|\mathbf Q_k\|_{\infty} \leq 1$ and $\tau_k > \|\mathbf Q_k\|_2^2$, then the nonlinearity eq:weak_potential is nonnegative with a unique global minimum at the origin, and a gradient that is nonexpansive.

Figures (9)

  • Figure 1: The WCRR potential (top) and its gradient (bottom) match the difference of two Moreau envelopes of the absolute value. Vertical ticks indicate the spline knots.
  • Figure 2: Learned filters on the denoising task with $K = 15$ and $d = 4$. The black lines split the filters into the 15 different groups of 4.
  • Figure 3: A pair of learned filters from the $\left(K = 30 \text{ and } d = 2\right)$ model. In the top row, a red frame highlights a location where the two filters differ most markedly. The magnitude of the Fourier transforms are shown in the bottom row.
  • Figure 4: Denoising results for $\sigma = 0.2$, shown on the full image and on a zoomed-in textured region. The reported metrics are (PSNR, SSIM).
  • Figure 5: Learned potential function and the corresponding filters. The axes correspond to the responses of the two filters.
  • ...and 4 more figures

Theorems & Definitions (2)

  • Theorem 1
  • Theorem 2