Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

About optimal loss function for training physics-informed neural networks under respecting causality

Vasiliy A. Es'kin, Danil V. Davydov, Ekaterina D. Egorova, Alexey O. Malkhanov, Mikhail A. Akhukov, Mikhail E. Smorkalov

TL;DR

A method is presented that allows to reduce a problem described by differential equations with initial and boundary conditions to the problem described only by differential equations, thus eliminating the need to tune the scaling coefficients for the terms related to boundary and initial conditions.

Abstract

A method is presented that allows to reduce a problem described by differential equations with initial and boundary conditions to the problem described only by differential equations. The advantage of using the modified problem for physics-informed neural networks (PINNs) methodology is that it becomes possible to represent the loss function in the form of a single term associated with differential equations, thus eliminating the need to tune the scaling coefficients for the terms related to boundary and initial conditions. The weighted loss functions respecting causality were modified and new weighted loss functions based on generalized functions are derived. Numerical experiments have been carried out for a number of problems, demonstrating the accuracy of the proposed methods.

About optimal loss function for training physics-informed neural networks under respecting causality

TL;DR

A method is presented that allows to reduce a problem described by differential equations with initial and boundary conditions to the problem described only by differential equations, thus eliminating the need to tune the scaling coefficients for the terms related to boundary and initial conditions.

Abstract

A method is presented that allows to reduce a problem described by differential equations with initial and boundary conditions to the problem described only by differential equations. The advantage of using the modified problem for physics-informed neural networks (PINNs) methodology is that it becomes possible to represent the loss function in the form of a single term associated with differential equations, thus eliminating the need to tune the scaling coefficients for the terms related to boundary and initial conditions. The weighted loss functions respecting causality were modified and new weighted loss functions based on generalized functions are derived. Numerical experiments have been carried out for a number of problems, demonstrating the accuracy of the proposed methods.
Paper Structure (25 sections, 70 equations, 7 figures, 6 tables, 2 algorithms)

This paper contains 25 sections, 70 equations, 7 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Formulation of the problem
  3. Physical-informed neural networks (PINNs)
  4. Reformulation of the problem
  5. Problem in time domain
  6. Optimization problem reformulation
  7. MAE loss
  8. Problem in spatial-temporal domain
  9. Causal training for physics-informed neural network
  10. Modification of the causal training for physics-informed neural network
  11. Alternatives of the causal training for physics-informed neural network
  12. Heaviside--weighted residual loss
  13. Dirac--weighted residual loss
  14. Expanding of the causal training approach to the spatial-temporal domain
  15. Training procedure
  16. ...and 10 more sections

Figures (7)

  • Figure 1: Schematic diagram of the general PINN method.
  • Figure 2: Allen--Cahn equation. (a) is reference solution, (b) is prediction of a trained physics-informed neural network based on algorithm \ref{['alg1']}, (c) is absolute difference between reference solution and predicted solution. The relative error $\epsilon_{\rm error}$ is $6.29\times 10^{-5}$. (d), (e) and (f) are comparison of the predicted (red dash lines) and reference solutions (blue solid lines) corresponding to three temporal snapshots at $t = 0.0$, $t=0.5$ and $t=1.0$, respectively.
  • Figure 3: Korteweg--De Vries equation. (a) is reference solution, (b) is prediction of a trained physics-informed neural network based on algorithm \ref{['alg2']}, (c) is the absolute difference between reference solution and predicted one. The relative error $\epsilon_{\rm error}$ is $6.84\times 10^{-3}$. (d), (e) and (f) are comparison of the predicted (red dash lines) and reference solutions (blue solid lines) corresponding to the three temporal snapshots at $t = 0.0$, $t=0.5$ and $t=1.0$, respectively.
  • Figure 4: Korteweg--De Vries equation. (a) is reference solution, (b) is prediction of a trained physics-informed neural network based on the algorithm \ref{['alg2']}, (c) is the absolute difference between reference solution and predicted solution. The relative error $\epsilon_{\rm error}$ is $2.45\times 10^{-3}$. (d), (e) and (f) are comparison of the predicted (red dash lines) and reference solutions (blue solid lines) corresponding to the three temporal snapshots at $t = 0.0$, $t=0.5$ and $t=1.0$, respectively. (g) is schematic representation of the architecture of the neural network.
  • Figure 5: Korteweg--De Vries equation. (a) is reference solution, (b) is prediction of a trained physics-informed neural network based on algorithm \ref{['alg1']} (best result for MLP, $\epsilon_{\rm error} = 6.84\times 10^{-3}$ for $t \in [0,1]$ and $\epsilon_{\rm error} = 6.61\times 10^{-1}$ for $t \in [0,2]$), (c) is absolute value of difference of reference solution and predicted solution. (d)--(f) are same for the PAF MLP ($\epsilon_{\rm error}=2.45\times 10^{-3}$ for $t \in [0,1]$ and $\epsilon_{\rm error} = 3.28\times 10^{-1}$ for $t \in [0,2]$).
  • ...and 2 more figures