Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Explainable Learning Based Regularization of Inverse Problems

Martin Burger, Samira Kabri, Gitta Kutyniok, Yunseok Lee, Lukas Weigand

TL;DR

The chapter develops a rigorous framework for explainable, data-driven regularization in inverse problems, focusing on spectral and frame-based approaches that generalize classical regularization theory. It analyzes how to learn regularization operators via supervised, adversarial, and self-supervised paradigms, deriving explicit optimal coefficients and proving convergence and rate results under mild probabilistic assumptions. A key contribution is showing that adversarial training can act as convergent regularization, with explicit formulations for infinite-norm adversaries and plug-and-play priors that connect to variational formulations. The work further bridges theory and practice by extending to diagonal frame decompositions and by exploring sparse, local CNN architectures that resemble TV-like regularization, providing insights into robustness and structure in learned regularizers.

Abstract

Machine learning techniques for the solution of inverse problems have become an attractive approach in the last decade, while their theoretical foundations are still in their infancy. In this chapter we want to pursue the study of regularization properties, robustness, convergence rates, and structure of regularizers for inverse problems obtained from different learning paradigms. For this sake we study simple architectures that are explainable in the sense that they allow for a theoretical analysis also in the infinite-dimensional limit. In particular we will advance the study of spectral architectures with new results on convergence rates highlighting the role of the smoothness in the training data set, and a study of adversarial robustness. We can show that adversarial training is actually a convergent regularization method. Moreover, we discuss extensions to frame systems and CNN-type architectures for variational regularizers, where we obtain some results on their structure by carefully designed numerical experiments.

Explainable Learning Based Regularization of Inverse Problems

TL;DR

The chapter develops a rigorous framework for explainable, data-driven regularization in inverse problems, focusing on spectral and frame-based approaches that generalize classical regularization theory. It analyzes how to learn regularization operators via supervised, adversarial, and self-supervised paradigms, deriving explicit optimal coefficients and proving convergence and rate results under mild probabilistic assumptions. A key contribution is showing that adversarial training can act as convergent regularization, with explicit formulations for infinite-norm adversaries and plug-and-play priors that connect to variational formulations. The work further bridges theory and practice by extending to diagonal frame decompositions and by exploring sparse, local CNN architectures that resemble TV-like regularization, providing insights into robustness and structure in learned regularizers.

Abstract

Machine learning techniques for the solution of inverse problems have become an attractive approach in the last decade, while their theoretical foundations are still in their infancy. In this chapter we want to pursue the study of regularization properties, robustness, convergence rates, and structure of regularizers for inverse problems obtained from different learning paradigms. For this sake we study simple architectures that are explainable in the sense that they allow for a theoretical analysis also in the infinite-dimensional limit. In particular we will advance the study of spectral architectures with new results on convergence rates highlighting the role of the smoothness in the training data set, and a study of adversarial robustness. We can show that adversarial training is actually a convergent regularization method. Moreover, we discuss extensions to frame systems and CNN-type architectures for variational regularizers, where we obtain some results on their structure by carefully designed numerical experiments.

Paper Structure

This paper contains 16 sections, 18 theorems, 147 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Learning paradigms for regularization methods in inverse problems
  3. Supervised learning
  4. Unsupervised learning with adversarial data
  5. Unsupervised generative learning
  6. Learning-based spectral regularizations
  7. Classical regularization theory
  8. Data-driven spectral regularization
  9. Self-supervised regularization with spectral plug-and-play priors
  10. Convergence rates of data-driven spectral regularizations
  11. Unsupervised data-driven regularization based on diagonal frame decompositions
  12. Regularization aspects of adversarial training
  13. $\infty$-Type adversaries
  14. Sparse Frame Architectures
  15. Sparse CNN Architectures
  16. ...and 1 more sections

Key Result

Theorem 1

For any $P_\varepsilon \in \Psi$, let there exist a constant $c > 0$ such that for $n$ large enough and $\Pi_n > 0$ for all $n \in {\mathbb N}$. Then, the family $\left \lbrace R^{\text{MSE}}_{ P_\varepsilon}\right\rbrace_{P_\varepsilon \in \Psi}$ is a convergent data-driven regularization on $\mathcal{D}(A^\dagger)\cap Y$ and over the fixed distribution $P_x$. In particular i and that

Figures (3)

  • Figure 1: Illustration of typical data (clean images on the left) and adversarial data (noisy images on the right).
  • Figure 2: Learned kernels $k^1,k^2,k^3$ in a multiresolution CNN architecture..
  • Figure 3: Learned kernels $k^1,k^2,k^3,k^4$ in a local CNN architecture.

Theorems & Definitions (47)

  • Definition 1: Regularization
  • Definition 2: Convergence
  • Definition 3: Data-driven regularization
  • Definition 4: Convergence
  • Theorem 1
  • proof
  • Theorem 2
  • proof
  • Theorem 3
  • proof
  • ...and 37 more