Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Shallow neural network yields regularization for ill-posed inverse problems

Lan Wang, Qiao Zhu, Bangti Jin, Ye Zhang

TL;DR

This work tackles ill-posed inverse problems $A(f)=g$ by integrating neural networks within a rigorous regularization framework. It establishes universal approximation results in Barron spaces for two-layer networks and introduces expanding neural network methods that use network width as a regularization parameter, with convergence guaranteed under Hölder continuity of $A$. It also develops a neural network–based Tikhonov scheme with convergence rates under variational source conditions, and demonstrates these ideas through numerical experiments showing regimes where small networks suffice under high noise and where regularization improves stability. The findings provide constructive strategies for designing neural-network-based regularizers with provable stability and convergence, and point to future work on deeper architectures, graph-based extensions, and efficient optimization. Overall, the paper bridges Barron-space theory and iterative/variational regularization to yield practical, theoretically grounded algorithms for ill-posed problems.

Abstract

In this paper, we establish universal approximation theorems for neural networks applied to general nonlinear ill-posed operator equations. In addition to the approximation error, the measurement error is also taken into account in our error estimation. We introduce the expanding neural network method as a novel iterative regularization scheme and prove its regularization properties under different a priori assumptions about the exact solutions. Within this framework, the number of neurons serves as both the regularization parameter and iteration number. We demonstrate that for data with high noise levels, a small network architecture is sufficient to obtain a stable solution, whereas a larger architecture may compromise stability due to overfitting. Furthermore, under standard assumptions in regularization theory, we derive convergence rate results for neural networks in the context of variational regularization. Several numerical examples are presented to illustrate the robustness of the proposed neural network-based algorithms.

Shallow neural network yields regularization for ill-posed inverse problems

TL;DR

This work tackles ill-posed inverse problems by integrating neural networks within a rigorous regularization framework. It establishes universal approximation results in Barron spaces for two-layer networks and introduces expanding neural network methods that use network width as a regularization parameter, with convergence guaranteed under Hölder continuity of . It also develops a neural network–based Tikhonov scheme with convergence rates under variational source conditions, and demonstrates these ideas through numerical experiments showing regimes where small networks suffice under high noise and where regularization improves stability. The findings provide constructive strategies for designing neural-network-based regularizers with provable stability and convergence, and point to future work on deeper architectures, graph-based extensions, and efficient optimization. Overall, the paper bridges Barron-space theory and iterative/variational regularization to yield practical, theoretically grounded algorithms for ill-posed problems.

Abstract

In this paper, we establish universal approximation theorems for neural networks applied to general nonlinear ill-posed operator equations. In addition to the approximation error, the measurement error is also taken into account in our error estimation. We introduce the expanding neural network method as a novel iterative regularization scheme and prove its regularization properties under different a priori assumptions about the exact solutions. Within this framework, the number of neurons serves as both the regularization parameter and iteration number. We demonstrate that for data with high noise levels, a small network architecture is sufficient to obtain a stable solution, whereas a larger architecture may compromise stability due to overfitting. Furthermore, under standard assumptions in regularization theory, we derive convergence rate results for neural networks in the context of variational regularization. Several numerical examples are presented to illustrate the robustness of the proposed neural network-based algorithms.

Paper Structure

This paper contains 12 sections, 10 theorems, 94 equations, 3 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Some properties of Barron spaces
  3. Expanding neural network methods
  4. Case 1: given an energy bound of $c_{\rho}(f^{\dagger})$.
  5. Case 2: without knowledge of $c_{\rho}(f^{\dagger})$
  6. A neural network-based Tikhonov regularization scheme
  7. Numerical experiments
  8. Conclusion and outlook
  9. Proof of Theorem \ref{['main']}
  10. Proof of Theorem \ref{['main2']}
  11. Proof of Theorem \ref{['well-posedness']}
  12. Proof of Theorem \ref{['convergencerate']}

Key Result

Proposition 1

(Weinan2019BarronSA) For any $1\le p\le \infty$, we have $\mathcal{B}_1=\mathcal{B}_{p}$ (as sets) and $\|f\|_{\mathcal{B}_1}=\|f\|_{\mathcal{B}_{p}}$.

Figures (3)

  • Figure 1: 1D linear integral equation: Relative $L^2$ error of $f_n^\delta$ obtained by Algorithm \ref{['alg:expanding_nn']} (top row), Algorithm \ref{['alg:modified_enn']} (middle row), and Tikhonov regularization (bottom row). Each column corresponds to a different random seed: 111, 666, 3333 (from left to right). The marker on each curve highlights the first network architecture that satisfied the stopping criterion for each noise level, with $\tau=1.0001$. The corresponding value of $n$ (the stopping number) is indicated in the legend.
  • Figure 2: Auto-convolution equation: Relative $L^2$ error of $f_n^{\delta}$ obtained by Algorithm \ref{['alg:expanding_nn']} (top row), Algorithm \ref{['alg:modified_enn']} (middle row), and Tikhonov regularization (bottom row). Each column corresponds to a different random seed: 678, 765, 987 (from left to right). The marker on each curve highlights the first network architecture that satisfied the stopping criterion for each noise level, with $\tau=1.0001$. The corresponding value of $n$ (the stopping number) is indicated in the legend. In some figures, the blue circles ($\delta=0.0001$) and green squares ($\delta=0.001$) coincide.
  • Figure 3: EIT: Relative $L^2$ error of $f_n^{\delta}$ obtained by Algorithm \ref{['alg:expanding_nn']} (top row), Algorithm \ref{['alg:modified_enn']} (middle row), and Tikhonov regularization (bottom row). Each column corresponds to a different random seed: 20, 30, 40 (from left to right). The marker on each curve highlights the first network architecture that satisfied the stopping criterion for each noise level, with $\tau=1.0001$. The corresponding value of $n$ (the stopping number) is indicated in the legend. In some figures, the blue circles ($\delta=0.0001$) and green squares ($\delta=0.001$) coincide.

Theorems & Definitions (24)

  • Definition 1
  • Proposition 1
  • Proposition 2: Properties of Barron space $\mathcal{B}_1$
  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Proposition 3: Approximation theorem
  • Definition 2
  • Theorem 1
  • ...and 14 more