Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Let data talk: data-regularized operator learning theory for inverse problems

Ke Chen, Chunmei Wang, Haizhao Yang

TL;DR

It is demonstrated that training a neural network on the regularized data is equivalent to supervised learning for a regularized inverse map, and sufficient conditions for the smoothness of such a regularization inverse map are provided.

Abstract

Regularization plays a pivotal role in integrating prior information into inverse problems. While many deep learning methods have been proposed to solve inverse problems, determining where to apply regularization remains a crucial consideration. Typical methods regularize neural networks via architecture, wherein neural network functions parametrize the parameter of interest or the regularization term. We introduce a novel approach, denoted as the "data-regularized operator learning" (DaROL) method, designed to address PDE inverse problems. The DaROL method trains a neural network on data, regularized through common techniques such as Tikhonov variational methods and Bayesian inference. The DaROL method offers flexibility across different frameworks, faster inverse problem-solving, and a simpler structure that separates regularization and neural network training. We demonstrate that training a neural network on the regularized data is equivalent to supervised learning for a regularized inverse map. Furthermore, we provide sufficient conditions for the smoothness of such a regularized inverse map and estimate the learning error in terms of neural network size and the number of training samples.

Let data talk: data-regularized operator learning theory for inverse problems

TL;DR

It is demonstrated that training a neural network on the regularized data is equivalent to supervised learning for a regularized inverse map, and sufficient conditions for the smoothness of such a regularization inverse map are provided.

Abstract

Regularization plays a pivotal role in integrating prior information into inverse problems. While many deep learning methods have been proposed to solve inverse problems, determining where to apply regularization remains a crucial consideration. Typical methods regularize neural networks via architecture, wherein neural network functions parametrize the parameter of interest or the regularization term. We introduce a novel approach, denoted as the "data-regularized operator learning" (DaROL) method, designed to address PDE inverse problems. The DaROL method trains a neural network on data, regularized through common techniques such as Tikhonov variational methods and Bayesian inference. The DaROL method offers flexibility across different frameworks, faster inverse problem-solving, and a simpler structure that separates regularization and neural network training. We demonstrate that training a neural network on the regularized data is equivalent to supervised learning for a regularized inverse map. Furthermore, we provide sufficient conditions for the smoothness of such a regularized inverse map and estimate the learning error in terms of neural network size and the number of training samples.
Paper Structure (24 sections, 11 theorems, 86 equations, 1 table)

This paper contains 24 sections, 11 theorems, 86 equations, 1 table.

Table of Contents

  1. Introduction
  2. Main Contributions.
  3. Organization.
  4. Notations.
  5. DaROL: data-regularized operator learning
  6. Preliminaries
  7. Regularization of the training data
  8. Proposed method
  9. Tikhonov regularized data
  10. Bayesian regularized data
  11. Learning error analysis
  12. Approximation error
  13. Generalization error
  14. Appendix
  15. SVD of submatrices
  16. ...and 9 more sections

Key Result

Lemma 1

For $(b,\lambda) \in \mathbb{R}^n \times \mathbb{R}_+$, let $\hat{x}$ be a solution to LASSO problem eqn:lasso with $I\coloneqq \text{supp}(\hat{x})$. If Assumption assum:nondegeneracy holds for $\hat{x}$, then the mapping is locally Lipschitz at the point $(b,\lambda)$ with a Lipschitz constant Here $\Bar{r} \coloneqq A\hat{x} - b$ represents the residual, and $\sigma_\text{min}(A_I)$ and $\sig

Theorems & Definitions (28)

  • Definition 2.1
  • Definition 2.2: ReLU Neural Network Function Class
  • Definition 2.3: ReLU Neural Network Operator Class
  • Remark
  • Lemma 1: Theorem 4.13 (b) in berk2022lasso
  • Theorem 1
  • proof
  • Remark
  • Theorem 2
  • Theorem 3: Theorem 1.1 of shen2019deep
  • ...and 18 more