Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generalization Bounds for Physics-Informed Neural Networks for the Incompressible Navier-Stokes Equations

Sebastien Andre-Sloan, Dibyakanti Kumar, Alejandro F Frangi, Anirbit Mukherjee

Abstract

This work establishes rigorous first-of-its-kind upper bounds on the generalization error for the method of approximating solutions to the (d+1)-dimensional incompressible Navier-Stokes equations by training depth-2 neural networks trained via the unsupervised Physics-Informed Neural Network (PINN) framework. This is achieved by bounding the Rademacher complexity of the PINN risk. For appropriately weight bounded net classes our derived generalization bounds do not explicitly depend on the network width and our framework characterizes the generalization gap in terms of the fluid's kinematic viscosity and loss regularization parameters. In particular, the resulting sample complexity bounds are dimension-independent. Our generalization bounds suggest using novel activation functions for solving fluid dynamics. We provide empirical validation of the suggested activation functions and the corresponding bounds on a PINN setup solving the Taylor-Green vortex benchmark.

Generalization Bounds for Physics-Informed Neural Networks for the Incompressible Navier-Stokes Equations

Abstract

This work establishes rigorous first-of-its-kind upper bounds on the generalization error for the method of approximating solutions to the (d+1)-dimensional incompressible Navier-Stokes equations by training depth-2 neural networks trained via the unsupervised Physics-Informed Neural Network (PINN) framework. This is achieved by bounding the Rademacher complexity of the PINN risk. For appropriately weight bounded net classes our derived generalization bounds do not explicitly depend on the network width and our framework characterizes the generalization gap in terms of the fluid's kinematic viscosity and loss regularization parameters. In particular, the resulting sample complexity bounds are dimension-independent. Our generalization bounds suggest using novel activation functions for solving fluid dynamics. We provide empirical validation of the suggested activation functions and the corresponding bounds on a PINN setup solving the Taylor-Green vortex benchmark.
Paper Structure (23 sections, 8 theorems, 82 equations, 2 figures)

This paper contains 23 sections, 8 theorems, 82 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Summary of Results
  3. Organization
  4. Empirical Evidence of Solving the Navier-Stokes PDE via Deep-Learning
  5. Unsupervised Ways of Solving Navier-Stokes with Neural Networks
  6. Semi-supervised Ways of Solving Navier-Stokes with Neural Networks
  7. Fully Data-Driven Ways of Solving Navier-Stokes with Neural Networks
  8. Motivations for Studying the Rademacher Complexity
  9. Mathematical Setup
  10. PINN Loss for d+1-Navier-Stokes
  11. Main Result on Generalization Error for $(d+1)$-Navier-Stokes
  12. Generalization Bound for Solving the Navier-Stokes PDE (Theorem \ref{['thm:NSRad']})
  13. An Outline of the Proof Technique
  14. Proof of Theorem \ref{['thm:NSRad']}
  15. Lemmas Towards the Generalization Bound for the Navier-Stokes PDE
  16. ...and 8 more sections

Key Result

Theorem 1

Consider neural networks ${\mathcal{N}}_{\bm{w}}$ belonging to a class of depth-2 neural networks. For the PINN empirical risk corresponding to the $(d+1)$-dimensional Navier–Stokes equations, denoted by $\hat{R}({\mathcal{N}}_{\bm{w}}, {\mathcal{S}}_n)$, where ${\mathcal{S}}_n$ represents the train where $N_r$ denotes the number of training data points in the domain and $N_0$ the number of bounda

Figures (2)

  • Figure 1: The left column is a snapshot of the true solution ($t=0$ for the left plot and $t=1$ for the right plot) and the right column is the neural net prediction. The rows represent the three components of the solution, $u,v$, and $p$. The domain is $(x,y)\in[0,2]^2$, with the viscosity set to $10^{-2}$ and the density set to $1$.
  • Figure 2: A scatter plot showing the relation between the generalization error upperbound given in Theorem \ref{['thm:NSRad']} and the generalization error calculated after training of the $tanh^3$ activated PINNs. Each point is labeled by the $N_r/N_0$ ratio at which the net was trained. On the left, the viscosity is set to $\nu=0.001$ and on the right $\nu=0.01$.

Theorems & Definitions (22)

  • Theorem : Informal Statement of Theorem \ref{['thm:NSRad']}
  • Definition 1: Huber Loss
  • Definition 2: Neural Network for $(d+1)$-Navier-Stokes
  • Definition 3: Collocation Points for $(d+1)$-Navier-Stokes
  • Definition 4: Empirical Risk for $(d+1)$-Navier-Stokes
  • Definition 5: Loss Function for $(d+1)$-Navier-Stokes
  • Definition 6: Class of Weights for $(d+1)$-Navier-Stokes
  • Theorem 1: Rademacher Based Upper-bound for $(d+1)$-Navier-Stokes
  • Remark 1
  • Lemma 2
  • ...and 12 more