Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Gauss-Newton Natural Gradient Descent for Physics-Informed Computational Fluid Dynamics

Anas Jnini, Flavio Vella, Marius Zeinhofer

TL;DR

This work proposes Gauss-Newton's method in function space for the solution of the Navier-Stokes equations in the physics-informed neural network (PINN) framework and shows that given a suitable integral discretization, the proposed optimization algorithm agrees with Gauss-Newton's method in parameter space.

Abstract

We propose Gauss-Newton's method in function space for the solution of the Navier-Stokes equations in the physics-informed neural network (PINN) framework. Upon discretization, this yields a natural gradient method that provably mimics the function space dynamics. Our computational results demonstrate close to single-precision accuracy measured in relative $L^2$ norm on a number of benchmark problems. To the best of our knowledge, this constitutes the first contribution in the PINN literature that solves the Navier-Stokes equations to this degree of accuracy. Finally, we show that given a suitable integral discretization, the proposed optimization algorithm agrees with Gauss-Newton's method in parameter space. This allows a matrix-free formulation enabling efficient scalability to large network sizes.

Gauss-Newton Natural Gradient Descent for Physics-Informed Computational Fluid Dynamics

TL;DR

This work proposes Gauss-Newton's method in function space for the solution of the Navier-Stokes equations in the physics-informed neural network (PINN) framework and shows that given a suitable integral discretization, the proposed optimization algorithm agrees with Gauss-Newton's method in parameter space.

Abstract

We propose Gauss-Newton's method in function space for the solution of the Navier-Stokes equations in the physics-informed neural network (PINN) framework. Upon discretization, this yields a natural gradient method that provably mimics the function space dynamics. Our computational results demonstrate close to single-precision accuracy measured in relative norm on a number of benchmark problems. To the best of our knowledge, this constitutes the first contribution in the PINN literature that solves the Navier-Stokes equations to this degree of accuracy. Finally, we show that given a suitable integral discretization, the proposed optimization algorithm agrees with Gauss-Newton's method in parameter space. This allows a matrix-free formulation enabling efficient scalability to large network sizes.
Paper Structure (33 sections, 2 theorems, 45 equations, 11 figures, 10 tables, 1 algorithm)

This paper contains 33 sections, 2 theorems, 45 equations, 11 figures, 10 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Physics-Informed Neural Networks (PINNs)
  3. PINN Optimization
  4. The Navier-Stokes Equations
  5. A Function-Space View on Neural Network Training
  6. Main Contributions
  7. Related Work
  8. Preliminaries
  9. Notation
  10. Physics-Informed Neural Networks
  11. Hard-Constraints in PINNs
  12. Function Space Optimization
  13. Second-Order Methods
  14. Gauss-Newton in Function Space
  15. Discretization in Tangent Space
  16. ...and 18 more sections

Key Result

Theorem 3.2

Assume that we employ algorithm eq:generic_second_order using the matrix eq:the_G producing a sequence of neural networks $(u_{\theta_k})$ and $(p_{\psi_k})$. Then it holds where $\Pi_{k}$ denotes the orthogonal projection onto the tangent space eq:tangent_space with respect to the inner product $g(u_k,p_k)$. The term $\epsilon_k$ corresponds to an error vanishing quadratically in the step and st

Figures (11)

  • Figure 1: Shown are the error $u_\theta - u^*$ and the push forwards of the Gauss-Newton and Euclidean gradient for the first component of the Kovasznay flow example in Section \ref{['sec:kovasznay']}. Note that the error $u_\theta-u^*$ is the optimal update direction and is closely matched by the Gauss-Newton direction. All plots are normed to lie in $[-1,1]$.
  • Figure 2: Median mean relative $L^2$ errors $E_{m}$ during training for the Kovasznay flow. Statistics are computed over 10 different initializations with the shaded area displaying the region between the first and third quartile.
  • Figure 3: Median mean relative $L^2$ errors $E_{m}$ during training for the Beltrami flow. Statistics are computed over 10 different initializations with the shaded area displaying the region between the first and third quartile.
  • Figure 4: Median mean relative $L^2$ errors $E_{m}$ during training for the Taylor-Green vortex. Statistics are computed over 10 different initializations with the shaded area displaying the region between the first and third quartile.
  • Figure 5: Visualization of the update directions of different optimizers in the example of the Kovasznay flow at the 20th iteration of a Gauss-Newton solve. Note that the error is the optimal update direction. The first component of the velocity is shown and all plots are normed to lie in $[-1,1]$.
  • ...and 6 more figures

Theorems & Definitions (7)

  • Remark 3.1: Galerkin Discretization
  • Theorem 3.2: Interpretation of Update Direction
  • proof
  • Remark 3.3
  • Remark 3.4
  • Proposition 2.1
  • proof