Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Slow Invariant Manifolds of Singularly Perturbed Systems via Physics-Informed Machine Learning

Dimitrios G. Patsatzis, Gianluca Fabiani, Lucia Russo, Constantinos Siettos

TL;DR

The proposed PIML scheme provides approximations, of equivalent or even higher accuracy, than those provided by other traditional GSPT-based methods, and importantly, for any practical purposes, it is not affected by the magnitude of the perturbation parameter.

Abstract

We present a physics-informed machine-learning (PIML) approach for the approximation of slow invariant manifolds (SIMs) of singularly perturbed systems, providing functionals in an explicit form that facilitate the construction and numerical integration of reduced order models (ROMs). The proposed scheme solves a partial differential equation corresponding to the invariance equation (IE) within the Geometric Singular Perturbation Theory (GSPT) framework. For the solution of the IE, we used two neural network structures, namely feedforward neural networks (FNNs), and random projection neural networks (RPNNs), with symbolic differentiation for the computation of the gradients required for the learning process. The efficiency of our PIML method is assessed via three benchmark problems, namely the Michaelis-Menten, the target mediated drug disposition reaction mechanism, and the 3D Sel'kov model. We show that the proposed PIML scheme provides approximations, of equivalent or even higher accuracy, than those provided by other traditional GSPT-based methods, and importantly, for any practical purposes, it is not affected by the magnitude of the perturbation parameter. This is of particular importance, as there are many systems for which the gap between the fast and slow timescales is not that big, but still ROMs can be constructed. A comparison of the computational costs between symbolic, automatic and numerical approximation of the required derivatives in the learning process is also provided.

Slow Invariant Manifolds of Singularly Perturbed Systems via Physics-Informed Machine Learning

TL;DR

The proposed PIML scheme provides approximations, of equivalent or even higher accuracy, than those provided by other traditional GSPT-based methods, and importantly, for any practical purposes, it is not affected by the magnitude of the perturbation parameter.

Abstract

We present a physics-informed machine-learning (PIML) approach for the approximation of slow invariant manifolds (SIMs) of singularly perturbed systems, providing functionals in an explicit form that facilitate the construction and numerical integration of reduced order models (ROMs). The proposed scheme solves a partial differential equation corresponding to the invariance equation (IE) within the Geometric Singular Perturbation Theory (GSPT) framework. For the solution of the IE, we used two neural network structures, namely feedforward neural networks (FNNs), and random projection neural networks (RPNNs), with symbolic differentiation for the computation of the gradients required for the learning process. The efficiency of our PIML method is assessed via three benchmark problems, namely the Michaelis-Menten, the target mediated drug disposition reaction mechanism, and the 3D Sel'kov model. We show that the proposed PIML scheme provides approximations, of equivalent or even higher accuracy, than those provided by other traditional GSPT-based methods, and importantly, for any practical purposes, it is not affected by the magnitude of the perturbation parameter. This is of particular importance, as there are many systems for which the gap between the fast and slow timescales is not that big, but still ROMs can be constructed. A comparison of the computational costs between symbolic, automatic and numerical approximation of the required derivatives in the learning process is also provided.
Paper Structure (25 sections, 3 theorems, 93 equations, 6 figures, 6 tables)

This paper contains 25 sections, 3 theorems, 93 equations, 6 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Geometric Singular Perturbation Theory and the Invariance Equation
  4. The proposed Physics-Informed Machine Learning (PIML) methodology
  5. Solution of the invariance equation with SLFNNs
  6. Solution of the invariance equation with RPNNs
  7. The sampling process
  8. The solution of the IE
  9. The benchmark problems
  10. The Michaelis-Menten mechanism
  11. The Target Mediated Drug Disposition mechanism
  12. The 3D Sel'kov model of glycolytic oscillations
  13. Numerical results
  14. The Michaelis-Menten mechanism
  15. The Target Mediated Drug Disposition mechanism
  16. ...and 10 more sections

Key Result

Lemma 1

Consider the MM slow subsystem in Eq. eq:MMss with $\epsilon \ll 1$. The analytic SIM approximation, derived on the basis of the invariance equation eq:Inv, is given by the $\mathcal{O}(\epsilon^2)$ regular asymptotic expansion: In addition, the explicit analytic SIM approximation, derived on the basis of CSP with one iteration, is:

Figures (6)

  • Figure 1: MM system (\ref{['eq:MMss']}). SIM surface of the slow system of Eq. \ref{['eq:MMss']} in the domain $y\in [10^{-6}, 1]$ for various values of $\epsilon \in [10^{-4}, 10^{-1}]$. The red trajectories (starting at red squares) are attracted to the SIM and then evolve on it.
  • Figure 2: TMDD system (\ref{['eq:TMDD_SPA']}). SIM surface of the slow system of Eq. \ref{['eq:TMDD_SPA']} in the domain $(y,z)\in [0.2, 2.0]\times[1.3,2.9]$ for three indicative values of $\epsilon$. The red trajectories (starting at red squares) are attracted to the SIM and then evolve on it until eventually exiting from its boundaries.
  • Figure 3: 3D Sel'kov system \ref{['eq:ToyLC_SPA']}. SIM surface of arising in the phase space along the domain of slow variables $(y,z)\in [0.2, 1.4]\times[0.3,2.1]$ for three indicative values of $\epsilon$. The red/black trajectories (starting at red/black squares) are attracted to the SIM and then evolve on it, in the exterior/interior of the limit cycle, towards reaching it.
  • Figure 4: MM system (\ref{['eq:MMss']}). Absolute errors (AE) of the SIM approximation in comparison to the numerical solution $x_{i,j}$ of the MM slow subsystem in Eq. \ref{['eq:MMss']}. Panels (a) and (b) depict the $\lvert x_{i,j} - \mathcal{N}(y_i,\epsilon_j) \rvert$ of the PIML schemes, while panels (c), (d), (e) and (f) depict the $\lvert x_{i,j} - h(y_i,\epsilon_j) \rvert$ of the sQSSA, GSPT $\mathcal{O}(\epsilon)$ and $\mathcal{O}(\epsilon^2)$ asymptotic expansions and the CSP with one iteration approximation in Eqs. (\ref{['eq:MManalSIM']}-\ref{['eq:MM_CSP']}), respectively.
  • Figure 5: TMDD system \ref{['eq:TMDD_SPA']}. Absolute errors (AE) of the SIM approximation in comparison to the numerical solution $x_{i,j}$ of the TMDD slow subsystem in Eq. \ref{['eq:TMDD_SPA']}. Panels (a) and (b) depict the $\lvert x_{i,j} - \mathcal{N}(\mathbf{y}_i,\epsilon_j) \rvert$ of the PIML schemes, while panels (c), (d), (e) and (f) depict the $\lvert x_{i,j} - h(\mathbf{y}_i,\epsilon_j) \rvert$ of the QSSA, GSPT $\mathcal{O}(\epsilon)$ and $\mathcal{O}(\epsilon^2)$ asymptotic expansions and the CSP with one iteration approximation in Eqs. (\ref{['eq:TMDD_GSPT']}, \ref{['eq:TMDD_CSP']}), respectively.
  • ...and 1 more figures

Theorems & Definitions (3)

  • Lemma 1
  • Lemma 2
  • Lemma 3