Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Numerical analysis of physics-informed neural networks and related models in physics-informed machine learning

Tim De Ryck, Siddhartha Mishra

TL;DR

A unified framework in which analysis of the various components of the error incurred by PINNs in approximating PDEs can be effectively carried out and identifies training errors as a key bottleneck which can adversely affect the overall performance of various models in physics-informed machine learning.

Abstract

Physics-informed neural networks (PINNs) and their variants have been very popular in recent years as algorithms for the numerical simulation of both forward and inverse problems for partial differential equations. This article aims to provide a comprehensive review of currently available results on the numerical analysis of PINNs and related models that constitute the backbone of physics-informed machine learning. We provide a unified framework in which analysis of the various components of the error incurred by PINNs in approximating PDEs can be effectively carried out. A detailed review of available results on approximation, generalization and training errors and their behavior with respect to the type of the PDE and the dimension of the underlying domain is presented. In particular, the role of the regularity of the solutions and their stability to perturbations in the error analysis is elucidated. Numerical results are also presented to illustrate the theory. We identify training errors as a key bottleneck which can adversely affect the overall performance of various models in physics-informed machine learning.

Numerical analysis of physics-informed neural networks and related models in physics-informed machine learning

TL;DR

A unified framework in which analysis of the various components of the error incurred by PINNs in approximating PDEs can be effectively carried out and identifies training errors as a key bottleneck which can adversely affect the overall performance of various models in physics-informed machine learning.

Abstract

Physics-informed neural networks (PINNs) and their variants have been very popular in recent years as algorithms for the numerical simulation of both forward and inverse problems for partial differential equations. This article aims to provide a comprehensive review of currently available results on the numerical analysis of PINNs and related models that constitute the backbone of physics-informed machine learning. We provide a unified framework in which analysis of the various components of the error incurred by PINNs in approximating PDEs can be effectively carried out. A detailed review of available results on approximation, generalization and training errors and their behavior with respect to the type of the PDE and the dimension of the underlying domain is presented. In particular, the role of the regularity of the solutions and their stability to perturbations in the error analysis is elucidated. Numerical results are also presented to illustrate the theory. We identify training errors as a key bottleneck which can adversely affect the overall performance of various models in physics-informed machine learning.
Paper Structure (44 sections, 43 theorems, 205 equations, 6 figures)

This paper contains 44 sections, 43 theorems, 205 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Physics-informed machine learning
  3. PDE setting
  4. Forward problems
  5. Inverse problems
  6. Ansatz spaces
  7. Linear models
  8. Nonlinear models
  9. Neural networks.
  10. Neural operators.
  11. Physics-informed residuals and loss functions
  12. Classical formulation
  13. Weak formulation
  14. Variational formulation
  15. Quadrature rules
  16. ...and 29 more sections

Key Result

Lemma 3.1

For any $\theta_1, \theta_2 \in \Theta$ and training set $\mathcal{S}$ it holds that

Figures (6)

  • Figure 6.1: Experimental results for the Navier-Stokes equation with $\nu=0.01$. The total error $\mathcal{E}$ and the training error $\mathcal{E}_T$ are shown in terms of the number of residual points $N_{\mathrm{int}}$ (left; and compared with the bound from Theorem \ref{['thm:generalization-ns']}) and also in terms of each other (right). Figure from deryck2021navierstokes.
  • Figure 6.2: Experimental results for the heat equation. The generalization error $\mathcal{E}_G$ and the training error $\mathcal{E}_T$ are shown in terms of the number of residual points $N_{\mathrm{int}}$ (left; and compared with the bound from Theorem \ref{['thm:generalization-heat']}) and also in terms of each other (right). Figure from molinaro_thesis.
  • Figure 6.3: Relative generalization error $\mathcal{E}_G$ (in percent) in terms of the parameter dimension $d$ for the heat equation with a parametrized initial condition. Figure from molinaro_thesis.
  • Figure 7.1: Poisson equation with Fourier features. Left: Optimal condition number vs. Number of Fourier features. Right: Training for the unpreconditioned and preconditioned Fourier features. Figure from deryck2023operator.
  • Figure 7.2: Linear advection equation with Fourier features. Left: Optimal condition number vs. $\beta$. Right: Training for the unpreconditioned and preconditioned Fourier features. Figure from deryck2023operator.
  • ...and 1 more figures

Theorems & Definitions (98)

  • Example 2.1: Semilinear heat equation
  • Example 2.2: Navier-Stokes equations
  • Example 2.3: Viscous and inviscid scalar conservation laws
  • Example 2.4: Poisson's equation
  • Example 2.5: Heat equation
  • Example 2.6: Poisson equation
  • Example 2.7: Stokes equation
  • Remark 2.8
  • Remark 2.9
  • Example 2.10: wPINNs for scalar conservation laws
  • ...and 88 more