Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convergence and error analysis of PINNs

Nathan Doumèche, Gérard Biau, Claire Boyer

TL;DR

This work analyzes the theoretical behavior of physics-informed neural networks (PINNs) in two regimes: hybrid modeling and PDE solvers. It first shows that vanilla PINNs can overfit and fail risk-consistency, motivating ridge regularization to achieve risk-consistency for linear and nonlinear PDEs. It then develops a Sobolev-regularization framework that yields well-posed weak formulations and, for linear PDEs, strong convergence results, including PDE-solver and hybrid-modeling cases with quantified physics-consistency. The results provide practical guidance on regularization choices, architecture considerations, and performance guarantees in PINN-based PDE solving and physics-informed learning. Overall, the paper links functional-analytic regularization to improved convergence and physics fidelity in PINNs, supporting reliable extrapolation and robust solution reconstruction.

Abstract

Physics-informed neural networks (PINNs) are a promising approach that combines the power of neural networks with the interpretability of physical modeling. PINNs have shown good practical performance in solving partial differential equations (PDEs) and in hybrid modeling scenarios, where physical models enhance data-driven approaches. However, it is essential to establish their theoretical properties in order to fully understand their capabilities and limitations. In this study, we highlight that classical training of PINNs can suffer from systematic overfitting. This problem can be addressed by adding a ridge regularization to the empirical risk, which ensures that the resulting estimator is risk-consistent for both linear and nonlinear PDE systems. However, the strong convergence of PINNs to a solution satisfying the physical constraints requires a more involved analysis using tools from functional analysis and calculus of variations. In particular, for linear PDE systems, an implementable Sobolev-type regularization allows to reconstruct a solution that not only achieves statistical accuracy but also maintains consistency with the underlying physics.

Convergence and error analysis of PINNs

TL;DR

This work analyzes the theoretical behavior of physics-informed neural networks (PINNs) in two regimes: hybrid modeling and PDE solvers. It first shows that vanilla PINNs can overfit and fail risk-consistency, motivating ridge regularization to achieve risk-consistency for linear and nonlinear PDEs. It then develops a Sobolev-regularization framework that yields well-posed weak formulations and, for linear PDEs, strong convergence results, including PDE-solver and hybrid-modeling cases with quantified physics-consistency. The results provide practical guidance on regularization choices, architecture considerations, and performance guarantees in PINN-based PDE solving and physics-informed learning. Overall, the paper links functional-analytic regularization to improved convergence and physics fidelity in PINNs, supporting reliable extrapolation and robust solution reconstruction.

Abstract

Physics-informed neural networks (PINNs) are a promising approach that combines the power of neural networks with the interpretability of physical modeling. PINNs have shown good practical performance in solving partial differential equations (PDEs) and in hybrid modeling scenarios, where physical models enhance data-driven approaches. However, it is essential to establish their theoretical properties in order to fully understand their capabilities and limitations. In this study, we highlight that classical training of PINNs can suffer from systematic overfitting. This problem can be addressed by adding a ridge regularization to the empirical risk, which ensures that the resulting estimator is risk-consistent for both linear and nonlinear PDE systems. However, the strong convergence of PINNs to a solution satisfying the physical constraints requires a more involved analysis using tools from functional analysis and calculus of variations. In particular, for linear PDE systems, an implementable Sobolev-type regularization allows to reconstruct a solution that not only achieves statistical accuracy but also maintains consistency with the underlying physics.
Paper Structure (28 sections, 32 theorems, 257 equations, 5 figures)

This paper contains 28 sections, 32 theorems, 257 equations, 5 figures.

Table of Contents

  1. Introduction
  2. The PINN framework
  3. PINNs can overfit
  4. Consistency of regularized PINNs for linear and nonlinear PDE systems
  5. Strong convergence of PINNs for linear PDE systems
  6. Sobolev regularization
  7. The PDE solver case
  8. The hybrid modeling case
  9. Mathematical details
  10. Some results of functional analysis on Lipschitz domains
  11. Some useful lemmas
  12. Proofs of Proposition \ref{['prop:densite']}
  13. Proofs of Section \ref{['POF']}
  14. Proof of Proposition \ref{['prop:friction']}
  15. Proof of Proposition \ref{['prop:wave']}
  16. ...and 13 more sections

Key Result

Proposition 2.3

Let $K \in \mathbb{N}$, $H\geqslant 2$, and $\Omega \subseteq \mathbb{R}^{d_1}$ be a bounded Lipschitz domain. Then $\mathrm{NN}_H:=\cup_{D}\mathrm{NN}_H(D)$ is dense in $(C^\infty(\bar{\Omega}, \mathbb{R}^{d_2}), \|\cdot\|_{C^K(\Omega)})$, i.e., for any function $u\in C^\infty(\bar{\Omega}, \mathbb

Figures (5)

  • Figure 1: Example of an inconsistent PINN estimator in hybrid modeling with $m = \gamma = 1$, $\varepsilon \sim \mathcal{N}(0, 10^{-2})$, and $n=10$.
  • Figure 2: Inconsistent PINN (left) compared to the solution $u^\star$ of the PDE (right) for the heat propagation case.
  • Figure 3: Regularized empirical risk (blue) and overfitting gap $\mathrm{OG}$ (orange) with respect to the number $p$ of epochs for $n=100$. The physics inconsistency $\mathrm{PI}(n)$ (green) and the $L^2$ error $\mathrm{err}(n)$ (red) are also depicted.
  • Figure 4: Distance $\mathrm{err}(n)$ to $u^\star$ (left) and physics inconsistency $\mathrm{PI}$ (right) of the regularized PINN estimator with respect to the number $n$ of observations in $\log$-$\log$ scale.
  • Figure 5: Functions $u_{\textrm{model}}$ (left), $u^\star$ (middle), and regularized PINN estimator with $n=10^3$ (right).

Theorems & Definitions (68)

  • Example 2.1: Maxwell equations
  • Example 2.2: Spatio-temporal condition function
  • Proposition 2.3: Density of neural networks in Hölder spaces
  • Proposition 3.1: Overfitting
  • Proposition 3.2: PDE solver overfitting
  • Definition 4.1: Ridge PINNs
  • Proposition 4.2: Bounding the norm of a neural network by the norm of its parameter
  • Example 4.3: Navier-Stokes equations
  • Definition 4.4: Polynomial operator
  • Definition 4.5: Degree
  • ...and 58 more