Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning sparsity-promoting regularizers for linear inverse problems

Giovanni S. Alberti, Ernesto De Vito, Tapio Helin, Matti Lassas, Luca Ratti, Matteo Santacesaria

TL;DR

The work addresses learning sparsity-promoting regularizers for linear inverse problems by formulating a bilevel optimization to select a synthesis operator $B$ that induces sparse representations of the solution. The inner problem enforces data fidelity with an $\ell^1$ sparsity penalty in the $B$-domain, producing $\hat{x}_B=B\hat{u}_B$, while the outer objective minimizes an expected reconstruction error via $L(B)=\mathbb{E}[\|R_B(y)-x\|_X^2]$. The authors establish well-posedness and stability for fixed $B$, provide finite-sample generalization bounds for learning $B$ from data, and demonstrate two concrete parameter classes: compact perturbations of a known operator and learning the mother wavelet, each with explicit covering-number bounds. The results enable data-driven incorporation of prior structure into infinite-dimensional sparsity-promoting regularization, with proven guarantees on existence, uniqueness, stability, and sample complexity. Overall, the framework blends variational regularization, operator learning, and statistical learning theory to promote sparse reconstructions in linear inverse problems.

Abstract

This paper introduces a novel approach to learning sparsity-promoting regularizers for solving linear inverse problems. We develop a bilevel optimization framework to select an optimal synthesis operator, denoted as $B$, which regularizes the inverse problem while promoting sparsity in the solution. The method leverages statistical properties of the underlying data and incorporates prior knowledge through the choice of $B$. We establish the well-posedness of the optimization problem, provide theoretical guarantees for the learning process, and present sample complexity bounds. The approach is demonstrated through examples, including compact perturbations of a known operator and the problem of learning the mother wavelet, showcasing its flexibility in incorporating prior knowledge into the regularization framework. This work extends previous efforts in Tikhonov regularization by addressing non-differentiable norms and proposing a data-driven approach for sparse regularization in infinite dimensions.

Learning sparsity-promoting regularizers for linear inverse problems

TL;DR

The work addresses learning sparsity-promoting regularizers for linear inverse problems by formulating a bilevel optimization to select a synthesis operator that induces sparse representations of the solution. The inner problem enforces data fidelity with an sparsity penalty in the -domain, producing , while the outer objective minimizes an expected reconstruction error via . The authors establish well-posedness and stability for fixed , provide finite-sample generalization bounds for learning from data, and demonstrate two concrete parameter classes: compact perturbations of a known operator and learning the mother wavelet, each with explicit covering-number bounds. The results enable data-driven incorporation of prior structure into infinite-dimensional sparsity-promoting regularization, with proven guarantees on existence, uniqueness, stability, and sample complexity. Overall, the framework blends variational regularization, operator learning, and statistical learning theory to promote sparse reconstructions in linear inverse problems.

Abstract

This paper introduces a novel approach to learning sparsity-promoting regularizers for solving linear inverse problems. We develop a bilevel optimization framework to select an optimal synthesis operator, denoted as , which regularizes the inverse problem while promoting sparsity in the solution. The method leverages statistical properties of the underlying data and incorporates prior knowledge through the choice of . We establish the well-posedness of the optimization problem, provide theoretical guarantees for the learning process, and present sample complexity bounds. The approach is demonstrated through examples, including compact perturbations of a known operator and the problem of learning the mother wavelet, showcasing its flexibility in incorporating prior knowledge into the regularization framework. This work extends previous efforts in Tikhonov regularization by addressing non-differentiable norms and proposing a data-driven approach for sparse regularization in infinite dimensions.

Paper Structure

This paper contains 12 sections, 13 theorems, 137 equations.

Table of Contents

  1. Problem formulation and main contributions
  2. Theoretical results on the deterministic optimization problem (1.2)
  3. Existence of the minimizer with fixed B in B
  4. Stability in B and uniqueness of the minimizer
  5. Statistical learning framework
  6. On the choice of the parameter class B
  7. Compact perturbations of a known operator
  8. Learning the mother wavelet
  9. Finite-dimensional setting
  10. White noise
  11. Connections with Dictionary Learning and unsupervised strategies
  12. Proofs of Section \ref{['sec:perturbation_known']}

Key Result

Theorem 2.3

Let $A \in \mathcal{L}(X,Y), \Sigma_{\varepsilon} \in \mathcal{L}(Y,Y)$, and $B \in \mathcal{L}(\ell^2,X)$ satisfy Assumptions ass:compatibility and ass:FBI, and let $y \in Y$. There exists a unique minimizer $\hat{u}_B = \hat{u}_B(y)$ of $J_B$, and, consequently, a unique $\hat{x}_B = B \hat{u}_B$

Theorems & Definitions (29)

  • Theorem 2.3
  • Theorem 2.5
  • Lemma 2.6
  • proof
  • Proposition 2.7
  • proof
  • proof : Proof of Thm. \ref{['thm:xhat_is_unique']} (existence)
  • Lemma 2.8
  • proof
  • Lemma 2.9
  • ...and 19 more