Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learned Regularization for Inverse Problems: Insights from a Spectral Model

Martin Burger, Samira Kabri

TL;DR

This work develops a spectral-regularization framework for data-driven inverse problems, formalizing how learned regularizers can converge to the generalized inverse $A^{\dagger}$ in infinite-dimensional spaces. It derives the mean-squared-error optimal spectral coefficients $g^{\text{mse}}_n(\mu,\pi)=\dfrac{\sigma_n}{\sigma_n^2+\Delta_n(\mu)/\Pi_n(\pi)}$ and proves convergence of the corresponding learned reg-ularizer $R^{\text{mse}}_{\mu}$ under a parameter rule as the noise level $\delta$ vanishes, with training noise $\Delta_n(\mu(\delta))$ and problem noise $\Delta_n(\nu^\delta)$ constrained in specific ways. It then adapts this framework to plug-and-play denoising (post-processing and proximal-map) and adversarial regularization (with and without a source condition), showing that the same optimal mse-regularizer can be achieved by several approaches provided the training noise is chosen appropriately; white-noise training generally yields convergence under mild operator/data assumptions. The paper offers concrete continuity and convergence conditions, compares approaches, and supports the theory with numerical CT-style experiments, offering practical guidance for designing stable, data-driven regularizers in high-dimensional inverse problems.

Abstract

In this chapter we provide a theoretically founded investigation of state-of-the-art learning approaches for inverse problems from the point of view of spectral reconstruction operators. We give an extended definition of regularization methods and their convergence in terms of the underlying data distributions, which paves the way for future theoretical studies. Based on a simple spectral learning model previously introduced for supervised learning, we investigate some key properties of different learning paradigms for inverse problems, which can be formulated independently of specific architectures. In particular we investigate the regularization properties, bias, and critical dependence on training data distributions. Moreover, our framework allows to highlight and compare the specific behavior of the different paradigms in the infinite-dimensional limit.

Learned Regularization for Inverse Problems: Insights from a Spectral Model

TL;DR

This work develops a spectral-regularization framework for data-driven inverse problems, formalizing how learned regularizers can converge to the generalized inverse in infinite-dimensional spaces. It derives the mean-squared-error optimal spectral coefficients and proves convergence of the corresponding learned reg-ularizer under a parameter rule as the noise level vanishes, with training noise and problem noise constrained in specific ways. It then adapts this framework to plug-and-play denoising (post-processing and proximal-map) and adversarial regularization (with and without a source condition), showing that the same optimal mse-regularizer can be achieved by several approaches provided the training noise is chosen appropriately; white-noise training generally yields convergence under mild operator/data assumptions. The paper offers concrete continuity and convergence conditions, compares approaches, and supports the theory with numerical CT-style experiments, offering practical guidance for designing stable, data-driven regularizers in high-dimensional inverse problems.

Abstract

In this chapter we provide a theoretically founded investigation of state-of-the-art learning approaches for inverse problems from the point of view of spectral reconstruction operators. We give an extended definition of regularization methods and their convergence in terms of the underlying data distributions, which paves the way for future theoretical studies. Based on a simple spectral learning model previously introduced for supervised learning, we investigate some key properties of different learning paradigms for inverse problems, which can be formulated independently of specific architectures. In particular we investigate the regularization properties, bias, and critical dependence on training data distributions. Moreover, our framework allows to highlight and compare the specific behavior of the different paradigms in the infinite-dimensional limit.
Paper Structure (16 sections, 8 theorems, 119 equations, 7 figures, 1 table)

This paper contains 16 sections, 8 theorems, 119 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Convergent Data-Driven Regularization
  3. Reconstruction with linear operators and spectral decomposition
  4. Statistical noise level
  5. Learned regularization methods
  6. Supervised Learning of Spectral Regularization
  7. Regularization by Denoising
  8. Post-processing approach
  9. Proximal map approach
  10. Adversarial Regularization
  11. Relaxed gradient penalty
  12. Promoting a source condition
  13. Comparison of all approaches
  14. Continuity
  15. Convergence
  16. ...and 1 more sections

Key Result

Lemma 1

The reconstruction operator $R^{\text{mse}}_{\mu}: Y \rightarrow X$ is continuous if and only if there exists a constant $c > 0$ such that for almost all $n \in \mathbb{N}$. In particular, this condition is fulfilled if there exists $c > 0$ and $n_0 \in \mathbb{N}$ such that for all $n \geq n_0$.

Figures (7)

  • Figure 1: Behavior of reconstruction error for the different approaches applied to different input dimensions. In all cases we choose $\epsilon, \|\epsilon\| = 0.001$ to be a perturbation parallel to $v_{\max}$, which denotes the singular vector that corresponds to the smallest singular value. The training noise which determines the data-driven reconstruction operators is chosen to be white Gaussian noise with $\boldsymbol{\delta}(\mu) = 0.001$.
  • Figure 2: Examples of noise samples with different decays of $\Delta_n(\nu)$.
  • Figure 3: Behavior of the reconstruction error obtained with the supervised and the adversarial approach. We test the performance on measurements corrupted with noise drawn from different kinds of noise models and decaying noise level.
  • Figure 4: Behavior of the reconstruction error obtained with the post-processing approach. We test the performance on measurements corrupted with noise drawn from different kinds of noise models and decaying noise level.
  • Figure 5: Reconstructions obtained with $R^{\text{mse}}_{\mu(\delta))}$ where $\Delta_n(\mu(\delta)) = \frac{\delta^2}{\sqrt{n}}$.
  • ...and 2 more figures

Theorems & Definitions (20)

  • Remark 1
  • Definition 1
  • Definition 2
  • Remark 2
  • Lemma 1: Continuity
  • proof
  • Theorem 1: Convergence
  • proof
  • Theorem 2
  • proof
  • ...and 10 more