Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Error estimates of residual minimization using neural networks for linear PDEs

Yeonjong Shin, Zhongqiang Zhang, George Em Karniadakis

TL;DR

The paper develops an abstract, operator-based convergence framework for residual-minimization methods using neural networks to solve linear PDEs, covering both strong and weak (PINN and variational) forms.It provides rigorous a priori and a posteriori error estimates for continuous RM and hp-VRM, and extends the analysis to discrete settings via discrete-norm relations and Rademacher complexity, ensuring convergence under specified assumptions.Two key strategies are used to handle discretization: norm-relations for stability and probabilistic bounds (Rademacher complexity) when Bernstein-type inequalities are unavailable, with hp-VRM offering a weak formulation that accommodates non-smooth networks.The framework is validated through illustrative examples in elliptic, advection-reaction, and fractional PDEs, and Appendices connect theory to practical verification via concrete problem classes.Overall, the work provides theoretical guidance for designing residual-based PINN and hp-VPINN loss functionals with convergence guarantees in linear PDE settings.

Abstract

We propose an abstract framework for analyzing the convergence of least-squares methods based on residual minimization when feasible solutions are neural networks. With the norm relations and compactness arguments, we derive error estimates for both continuous and discrete formulations of residual minimization in strong and weak forms. The formulations cover recently developed physics-informed neural networks based on strong and variational formulations.

Error estimates of residual minimization using neural networks for linear PDEs

TL;DR

The paper develops an abstract, operator-based convergence framework for residual-minimization methods using neural networks to solve linear PDEs, covering both strong and weak (PINN and variational) forms.It provides rigorous a priori and a posteriori error estimates for continuous RM and hp-VRM, and extends the analysis to discrete settings via discrete-norm relations and Rademacher complexity, ensuring convergence under specified assumptions.Two key strategies are used to handle discretization: norm-relations for stability and probabilistic bounds (Rademacher complexity) when Bernstein-type inequalities are unavailable, with hp-VRM offering a weak formulation that accommodates non-smooth networks.The framework is validated through illustrative examples in elliptic, advection-reaction, and fractional PDEs, and Appendices connect theory to practical verification via concrete problem classes.Overall, the work provides theoretical guidance for designing residual-based PINN and hp-VPINN loss functionals with convergence guarantees in linear PDE settings.

Abstract

We propose an abstract framework for analyzing the convergence of least-squares methods based on residual minimization when feasible solutions are neural networks. With the norm relations and compactness arguments, we derive error estimates for both continuous and discrete formulations of residual minimization in strong and weak forms. The formulations cover recently developed physics-informed neural networks based on strong and variational formulations.

Paper Structure

This paper contains 21 sections, 18 theorems, 80 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Assumptions
  3. Limitations and remarks
  4. Mathematical Setup and Preliminaries
  5. Solution of the problem and norm relations
  6. Loss Functionals of Residual Minimization
  7. Feed-forward Neural Networks
  8. Continuous Residual Minimization
  9. Error estimates
  10. The effect of discretizing the loss functionals
  11. Using discrete norm relations
  12. Using Rademacher complexity
  13. hp-Variational Residual Minimization
  14. Definition of hp-VRM
  15. Error estimates
  16. ...and 6 more sections

Key Result

Theorem 2.1

Suppose $(X,\|\cdot\|_V)$ is a dense subspace of $(V,\|\cdot\|_V)$ and let $A$ be a bounded linear operator from $(X,\|\cdot\|_V)$ to $(Y,\|\cdot\|_Y)$. Then, there exists a unique extension $\tilde{A}$ from $(V,\|\cdot\|_V)$ to $(Y,\|\cdot\|_Y)$ of $A$. That is, $Av = \tilde{A}v$ for all $v \in X$

Figures (1)

  • Figure 1: The sketch of critical concepts for convergence of continuous and discrete residual minimization using neural networks. Here we assume a zero training error.

Theorems & Definitions (52)

  • Remark 2.1
  • Definition 2.1
  • Theorem 2.1: Bounded Linear Transformation Theorem reed2012methods
  • Proposition 2.4: Uniqueness
  • proof
  • Remark 2.2
  • Remark 2.2
  • Remark 3.1
  • Remark 3.2
  • Proposition 3.1
  • ...and 42 more