Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Parsimonious Physics-Informed Random Projection Neural Networks for Initial-Value Problems of ODEs and index-1 DAEs

Gianluca Fabiani, Evangelos Galaris, Lucia Russo, Constantinos Siettos

Abstract

We address a physics-informed neural network based on the concept of random projections for the numerical solution of IVPs of nonlinear ODEs in linear-implicit form and index-1 DAEs, which may also arise from the spatial discretization of PDEs. The scheme has a single hidden layer with appropriately randomly parametrized Gaussian kernels and a linear output layer, while the internal weights are fixed to ones. The unknown weights between the hidden and output layer are computed by Newton's iterations, using the Moore-Penrose pseudoinverse for low to medium, and sparse QR decomposition with regularization for medium to large scale systems. To deal with stiffness and sharp gradients, we propose a variable step size scheme for adjusting the interval of integration and address a continuation method for providing good initial guesses for the Newton iterations. Based on previous works on random projections, we prove the approximation capability of the scheme for ODEs in the canonical form and index-1 DAEs in the semiexplicit form. The optimal bounds of the uniform distribution are parsimoniously chosen based on the bias-variance trade-off. The performance of the scheme is assessed through seven benchmark problems: four index-1 DAEs, the Robertson model, a model of five DAEs describing the motion of a bead, a model of six DAEs describing a power discharge control problem, the chemical Akzo Nobel problem and three stiff problems, the Belousov-Zhabotinsky, the Allen-Cahn PDE and the Kuramoto-Sivashinsky PDE. The efficiency of the scheme is compared with three solvers ode23t, ode23s, ode15s of the MATLAB ODE suite. Our results show that the proposed scheme outperforms the stiff solvers in several cases, especially in regimes where high stiffness or sharp gradients arise in terms of numerical accuracy, while the computational costs are for any practical purposes comparable.

Parsimonious Physics-Informed Random Projection Neural Networks for Initial-Value Problems of ODEs and index-1 DAEs

Abstract

We address a physics-informed neural network based on the concept of random projections for the numerical solution of IVPs of nonlinear ODEs in linear-implicit form and index-1 DAEs, which may also arise from the spatial discretization of PDEs. The scheme has a single hidden layer with appropriately randomly parametrized Gaussian kernels and a linear output layer, while the internal weights are fixed to ones. The unknown weights between the hidden and output layer are computed by Newton's iterations, using the Moore-Penrose pseudoinverse for low to medium, and sparse QR decomposition with regularization for medium to large scale systems. To deal with stiffness and sharp gradients, we propose a variable step size scheme for adjusting the interval of integration and address a continuation method for providing good initial guesses for the Newton iterations. Based on previous works on random projections, we prove the approximation capability of the scheme for ODEs in the canonical form and index-1 DAEs in the semiexplicit form. The optimal bounds of the uniform distribution are parsimoniously chosen based on the bias-variance trade-off. The performance of the scheme is assessed through seven benchmark problems: four index-1 DAEs, the Robertson model, a model of five DAEs describing the motion of a bead, a model of six DAEs describing a power discharge control problem, the chemical Akzo Nobel problem and three stiff problems, the Belousov-Zhabotinsky, the Allen-Cahn PDE and the Kuramoto-Sivashinsky PDE. The efficiency of the scheme is compared with three solvers ode23t, ode23s, ode15s of the MATLAB ODE suite. Our results show that the proposed scheme outperforms the stiff solvers in several cases, especially in regimes where high stiffness or sharp gradients arise in terms of numerical accuracy, while the computational costs are for any practical purposes comparable.
Paper Structure (22 sections, 5 theorems, 54 equations, 8 figures, 14 tables, 1 algorithm)

This paper contains 22 sections, 5 theorems, 54 equations, 8 figures, 14 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Methods
  3. Description of the Problem
  4. Physics-informed machine learning for the solution of differential equations
  5. Random Projection Neural Networks
  6. The Proposed Physics-Informed RPNN-based method for the solution of ODES and index-1 DAEs
  7. Approximation with the PIRPNN
  8. Computation of the unknown weights
  9. Parsimonious construction of the PIRPNN
  10. The variable step size scheme.
  11. A continuation method for Newton's iterations.
  12. Estimation of the interval of uniform distribution based on the variance/bias trade-off decomposition
  13. The algorithm
  14. Numerical Results
  15. Case Study 1: The Robertson index-1 DAEs
  16. ...and 7 more sections

Key Result

Theorem 2.1

SLFNNs as defined by Eq.(ARXIV_article:eq:randomexpansion) with weights $\boldsymbol{w}_{i}$ and biases $b_i$ selected randomly from a uniform distribution and for any family of $L^2$ integrable basis functions $\phi_i$, are universal approximators of any Lipschitz continuous function $f$ defined in is of the order of $(C/\sqrt{N})$, where $C=C(f,\phi_i, \beta, \Omega, \alpha, d)$; $\mathbb{E}$ de

Figures (8)

  • Figure 1: The numerical solution of the van der Pol problem with the proposed PIRPNN with $\mu=100$ in the interval $[0 \quad 3\mu]$ with respect to $c$ and $N$ for $n=20$; $RelTol$ and $AbsTol$ were set to 1e$-$06; (a) Sum of the Bias and Variance of the approximation with respect to the reference solution as obtained with ode15s with $AbsTol$ and $RelTol$ set at 1e$-$14, (b) Computational Times (s).
  • Figure 2: The Robertson index-1 DAEs \ref{['ARXIV_article:eq:robertson']}shampine1999solving. Approximate solutions computed in the interval $[0 \quad 4\cdot 10^{11}]$ with both absolute and relative tolerances set to 1e$-$03 ((a), (b) and (c)) and 1e$-$06 ((d), (e) and (f)). Insets depict zooms around the reference solution.
  • Figure 3: The mechanics problem of non autonomous index-1 DAEs \ref{['ARXIV_article:eq:ex11']}shampine1999solving. Approximate solutions computed in the interval $[0 \quad 15]$ with both absolute and relative tolerances set to 1e$-$03. Insets depict zooms around the reference solution.
  • Figure 4: Power discharge control non autonomous index-1 DAEs problem \ref{['ARXIV_article:eq:ex13']}shampine1999solving. Approximate solutions computed in the interval $[0 \quad 40]$ with both absolute and relative tolerances set to 1e$-$03. Insets depict zooms around the reference solution.
  • Figure 5: The chemical Akzo Nobel DAE problem \ref{['ARXIV_article:eq:chemakzo']}. Approximate solutions computed in the interval $[0 \quad 180]$ with both absolute and relative tolerances set to 1e$-$03. Insets depict zooms around the reference solution.
  • ...and 3 more figures

Theorems & Definitions (7)

  • Theorem 2.1
  • Theorem 2.2
  • Theorem 2.3
  • Proposition 1
  • proof
  • Proposition 2
  • proof