Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Incorporating Continuous Dependence Qualifies Physics-Informed Neural Networks for Operator Learning

Guojie Li, Wuyue Yang, Liu Hong

Abstract

Physics-informed neural networks (PINNs) have been proven as a promising way for solving various partial differential equations, especially high-dimensional ones and those with irregular boundaries. However, their capabilities in real applications are highly restricted by their poor generalization performance. Inspired by the rigorous mathematical statements on the well-posedness of PDEs, we develop a novel extension of PINNs by incorporating the additional information on the continuous dependence of PDE solutions with respect to parameters and initial/boundary values (abbreviated as cd-PINN). Extensive numerical experiments demonstrate that, with limited labeled data, cd-PINN achieves 1-3 orders of magnitude lower in test MSE than DeepONet and FNO. Therefore, incorporating the continuous dependence of PDE solutions provides a simple way for qualifying PINNs for operator learning.

Incorporating Continuous Dependence Qualifies Physics-Informed Neural Networks for Operator Learning

Abstract

Physics-informed neural networks (PINNs) have been proven as a promising way for solving various partial differential equations, especially high-dimensional ones and those with irregular boundaries. However, their capabilities in real applications are highly restricted by their poor generalization performance. Inspired by the rigorous mathematical statements on the well-posedness of PDEs, we develop a novel extension of PINNs by incorporating the additional information on the continuous dependence of PDE solutions with respect to parameters and initial/boundary values (abbreviated as cd-PINN). Extensive numerical experiments demonstrate that, with limited labeled data, cd-PINN achieves 1-3 orders of magnitude lower in test MSE than DeepONet and FNO. Therefore, incorporating the continuous dependence of PDE solutions provides a simple way for qualifying PINNs for operator learning.

Paper Structure

This paper contains 13 sections, 4 theorems, 36 equations, 9 figures, 1 table.

Table of Contents

  1. Introduction
  2. Results and Discussion
  3. Benchmarking Against State-of-the-Art Operator Learning Frameworks
  4. Effect of Residual Point Density
  5. Influence of PDE Dimensionality
  6. Comparison to FDM
  7. Applicability to Complex Systems
  8. Failure of Regularity
  9. Real Application: Protein Aggregation Dynamics in Alzheimer's Disease
  10. Methods
  11. Regularity Theory for Parabolic PDE
  12. PINN with Continuous Dependence (cd-PINN)
  13. Conclusion

Key Result

Theorem 1

(Maximum Principle for Cauchy Problem of Diffusion Equation) Suppose $u \in C^{2,1}(\Omega \times (0, T]) \cap C(\Omega \times [0, T])$ solves and satisfies the growth estimate for constants $A$, $a>0$. Then where $Q_T = \Omega \times (0, T)$, and $\partial_p Q_T = \Omega \times \{t=0\}$, $\Omega \subset \mathbb{R}^n$ is a bounded region, $\partial \Omega \in C^{\infty}$, and $T > 0$.

Figures (9)

  • Figure 1: Illustration of the idea, problem setup, and architecture of cd-PINN. (A) Schematic diagram of the input and output of neural operators. (B) MAD learns the solution of the equation under new configurations by fine-tuning the encoding $c$. (C) The objective function of cd-PINN is based on the continuity assumption. (D) Illustration of the labeled training data. For each output $u(t_i, \boldsymbol{x}_i)$, we require the same number of evaluations of $a(t_i, \boldsymbol{x}_i)$ and $u_0(\boldsymbol{x}_i)$ at the same scattered space-time point $(t_i, \boldsymbol{x}_i)$. (E) The flowchart for calculating residual loss. (F) The architecture of cd-PINN.
  • Figure 2: Results of the parameterized diffusion and wave equations. (A) The test MSE of cd-PINN, PINN, cd-PINN$^{\#}$, FNO, and DeepONet on the test dataset as the number of training epochs increases for the parameterized diffusion equation. (B) The NLMAE of predictions, (C) predictions on new configurations $\sigma=0.2, \mu=0.0$ without labeled training data, and (D) the test MSE as a function of the number of training epochs. (E) The NLMAE, (F) predictions, and (H) absolute errors of cd-PINN, PI-DeepONet, DeepONet, and FNO for parameterized wave equation at $c=0.505, k=0.505$. (G) 20 labeled training data points randomly selected from the low-resolution with $c=0.505, k=0.505$.
  • Figure 3: Results of the parameterized Poisson equation. (A) Comparison of the NRMSE between cd-PINN and PINN with respect to different numbers of residual points, where the vertical axis represents the number of $2^N$ residual data points used. (B) The test MSE and (C) $\mathcal{L}_{cd}$ term of cd-PINN and PINN as the number of training epochs changes when the number of residual points is fixed as $2^{11}$. (D) The NLMAE of predictions of cd-PINN and PINN. (E) 20 labeled training data points randomly selected from the low-resolution data when $a=2.45, b=2.45$. (F) The high-resolution true solution of the equation at new configurations $a=5.0, b=5.0$. (G) The predicted results and (H) absolute errors of cd-PINN and PINN at configurations $a=5.0, b=5.0$.
  • Figure 4: Results of the parameterized diffusion-reaction equation.(A), (B), and (C) show results for 2D, 5D, and 8D diffusion-reaction equations, respectively. For each dimension, from left to right: NLMAE of PINN and cd-PINN, test MSE, and $\mathcal{L}_{cd}$ versus training iterations.
  • Figure 5: Results of the parameterized Burgers equation.(A) The NLMAE of predictions of the cd-PINN model under each set of $(t, \nu)$. (B) NRMSE and epochs per second $v.s.$ number of residual points. The blue line represents the epochs per second, while the red line represents the NRMSE of the predicted results. (C) Comparison on the computational efficiency of cd-PINN and Newton-Implicit FDM. (D) The numerical solutions of Newton-Implicit FDM, (E) predicted solutions of cd-PINN as well as (F) absolute errors between the two methods at $\nu = 0.01, 0.03, 0.05, 0.07$ and $0.10$, respectively. (G) Comparison of cd-PINN predicted solutions and numerical solution obtained via the FDM at $t=0.0, 0.25, 0.50$ for various values of $\nu$.
  • ...and 4 more figures

Theorems & Definitions (5)

  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Theorem 4
  • proof