Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Universal Architectures for the Learning of Polyhedral Norms and Convex Regularizers

Michael Unser, Stanislas Ducotterd

TL;DR

This paper addresses the task of learning convex regularizers to guide the reconstruction of images from limited data by imposing that the reconstruction be amplitude-equivariant, and proposes an implementation that relies on a specific architecture that offers essentially the same convergence and robustness guarantees.

Abstract

This paper addresses the task of learning convex regularizers to guide the reconstruction of images from limited data. By imposing that the reconstruction be amplitude-equivariant, we narrow down the class of admissible functionals to those that can be expressed as a power of a seminorm. We then show that such functionals can be approximated to arbitrary precision with the help of polyhedral norms. In particular, we identify two dual parameterizations of such systems: (i) a synthesis form with an $\ell_1$-penalty that involves some learnable dictionary; and (ii) an analysis form with an $\ell_\infty$-penalty that involves a trainable regularization operator. After having provided geometric insights and proved that the two forms are universal, we propose an implementation that relies on a specific architecture (tight frame with a weighted $\ell_1$ penalty) that is easy to train. We illustrate its use for denoising and the reconstruction of biomedical images. We find that the proposed framework outperforms the sparsity-based methods of compressed sensing, while it offers essentially the same convergence and robustness guarantees.

Universal Architectures for the Learning of Polyhedral Norms and Convex Regularizers

TL;DR

This paper addresses the task of learning convex regularizers to guide the reconstruction of images from limited data by imposing that the reconstruction be amplitude-equivariant, and proposes an implementation that relies on a specific architecture that offers essentially the same convergence and robustness guarantees.

Abstract

This paper addresses the task of learning convex regularizers to guide the reconstruction of images from limited data. By imposing that the reconstruction be amplitude-equivariant, we narrow down the class of admissible functionals to those that can be expressed as a power of a seminorm. We then show that such functionals can be approximated to arbitrary precision with the help of polyhedral norms. In particular, we identify two dual parameterizations of such systems: (i) a synthesis form with an -penalty that involves some learnable dictionary; and (ii) an analysis form with an -penalty that involves a trainable regularization operator. After having provided geometric insights and proved that the two forms are universal, we propose an implementation that relies on a specific architecture (tight frame with a weighted penalty) that is easy to train. We illustrate its use for denoising and the reconstruction of biomedical images. We find that the proposed framework outperforms the sparsity-based methods of compressed sensing, while it offers essentially the same convergence and robustness guarantees.

Paper Structure

This paper contains 21 sections, 7 theorems, 43 equations, 4 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Variational Setup for the Resolution of Inverse Problems
  4. Equivariant Regularizers
  5. Universal Equivariant Convex Regularizers
  6. Properties of the Unit Regularization Ball
  7. Universal Parametric Polyhedral Regularizers
  8. Supporting Banach-Space Theory
  9. Nomenclature and notation
  10. Polyhedral Banach Spaces
  11. Parametric Reconstruction Pipelines
  12. Sparse Encoding in a Frame/Dictionary
  13. Universal Weighted $\ell_\infty$-Regularization
  14. Weighted $\ell_1$-Regularization Revisited
  15. Implementation of a Trainable Convex Regularizer
  16. ...and 6 more sections

Key Result

Theorem 3

Let $g:{\mathcal{X}} \to \mathbb{R}$ be a convex, symmetric (and continuous) functional on the Banach space ${\mathcal{X}}$. Then, the following statements are equivalent.

Figures (4)

  • Figure 1: Unit ball of a polyhedral norm: (a) Contour plot of ${\bf{x}} \mapsto \|{\bf{L}} {\bf{x}}\|_{\ell_1}$ with the darker central region in the plane being the unit ball $B_{p_1}$. (b) Overlay of the vertex (yellow) and facet (blue) vectors, the latter being the normals of the supporting hyperplanes of $B_{p_1}$.
  • Figure 1: Learned proximal nonlinearities in the $3 \times 3=9$ channel scenario.
  • Figure 2: The extreme points of a matrix ${\bf{G}}$ are given by the vertices of the symmetric convex hull of its column vectors represented as dark points on the left display.
  • Figure 2: Comparison of ground truth, zero-fill/backprojection ${\bf{H}}^{\mathsf{T}} {\bf{y}}$, and four variational reconstructions of the brain image from radially sampled Fourier data. Lower panel: zoom of a region of interest. The SNR is evaluated with respect to the groundtruth.

Theorems & Definitions (18)

  • Definition 1
  • Definition 2
  • Theorem 3
  • Proof 1
  • Definition 1
  • Theorem 2: Narici2010
  • Theorem 3: Dual parameterization of all polyhedral norms
  • Theorem 4: Universality of polyhedral norm approximations
  • Proof 2
  • Definition 1
  • ...and 8 more