Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A review of shape-morphing solutions and evolutional neural networks for spatiotemporal dynamics

Mohammad Farazmand

Abstract

Shape-morphing solutions (SMS) refer to a class of approximate solutions of partial differential equations (PDEs) with the distinguishing feature that they depend nonlinearly on a set of time-dependent parameters. They generalize Galerkin truncations by allowing the basis (or trial functions) to evolve in time in order to adapt to the solution of the PDE. As such, SMS are particularly suitable for reduced-order modeling as well as high fidelity simulation of multiscale systems which exhibit localized time-dependent features, such as vortices, dispersive wave packets, and shocks. Furthermore, being mesh-free, SMS is scalable for solving PDEs in higher spatial dimensions. As a special case, SMS allows the approximation of the PDE's solution by a neural network whose weights and biases depend on time. Such neural networks are known as evolutional neural networks or neural Galerkin schemes. The evolution of SMS parameters is dictated by the SMS equation, a set of ordinary differential equations derived from the Dirac-Frenkel variational principle. Over the past five years, contributions to the theory and computation of SMS have been growing rapidly. Here, we survey these developments, showcase some applications of SMS, and highlight important open problems for future research. At the same time, this review is structured to serve as a tutorial for applied mathematicians, physicist, and engineers who wish to enter this field.

A review of shape-morphing solutions and evolutional neural networks for spatiotemporal dynamics

Abstract

Shape-morphing solutions (SMS) refer to a class of approximate solutions of partial differential equations (PDEs) with the distinguishing feature that they depend nonlinearly on a set of time-dependent parameters. They generalize Galerkin truncations by allowing the basis (or trial functions) to evolve in time in order to adapt to the solution of the PDE. As such, SMS are particularly suitable for reduced-order modeling as well as high fidelity simulation of multiscale systems which exhibit localized time-dependent features, such as vortices, dispersive wave packets, and shocks. Furthermore, being mesh-free, SMS is scalable for solving PDEs in higher spatial dimensions. As a special case, SMS allows the approximation of the PDE's solution by a neural network whose weights and biases depend on time. Such neural networks are known as evolutional neural networks or neural Galerkin schemes. The evolution of SMS parameters is dictated by the SMS equation, a set of ordinary differential equations derived from the Dirac-Frenkel variational principle. Over the past five years, contributions to the theory and computation of SMS have been growing rapidly. Here, we survey these developments, showcase some applications of SMS, and highlight important open problems for future research. At the same time, this review is structured to serve as a tutorial for applied mathematicians, physicist, and engineers who wish to enter this field.
Paper Structure (21 sections, 1 theorem, 59 equations, 8 figures, 1 table)

This paper contains 21 sections, 1 theorem, 59 equations, 8 figures, 1 table.

Table of Contents

  1. Introduction
  2. Historical background
  3. Outline
  4. Working example
  5. Shape-morphing solutions
  6. Examples of SMS
  7. Evolution of SMS
  8. Enforcing conserved quantities
  9. Geometric interpretation of SMS
  10. Neural networks as SMS
  11. Numerical examples
  12. Nonlinear Schrödinger
  13. Navier--Stokes
  14. Kuramoto–Sivashinsky
  15. Fokker--Planck
  16. ...and 6 more sections

Key Result

Theorem 1

Under Assumption ass:linInd, the minimization problem eq:min_res has a unique solution which satisfies, where the metric tensor $M:\Omega\to\mathbb R^{n\times n}$ and the vector field $\mathbf f:\Omega\to\mathbb R^n$ are defined by for $i,j\in\{1,2,\cdots,n\}$.

Figures (8)

  • Figure 1: Three examples of shape-morphing solutions solving (a) the nonlinear Schrödinger equation, (b) the Kuramoto--Sivashinsky equation, (c) the Fokker--Planck equation. These results are discussed in detail in Section \ref{['sec:num_ex']}.
  • Figure 2: Illustrating the SMS manifold as a graph over its parameters. Here $\hat{u}(t)$ is shorthand for $\hat{u}(\cdot,\pmb\theta(t))$.
  • Figure 3: Architecture of a shallow evolutional neural network with $r$ nodes. The network parameters $\pmb\theta(t)=\{\alpha_i(t),\mathbf w_i(t),b_i(t)\}_{i=1}^r$ are all time-dependent.
  • Figure 4: Comparison for the NLS equation. The time series shows $|u(0,t)|$, the wave height at $x=0$. The SMS outperforms a comparable POD-based reduced-order model.
  • Figure 5: Leapfrogging dynamics of vortices reproduced by an SMS with $n=16$ parameters.
  • ...and 3 more figures

Theorems & Definitions (10)

  • Definition 1: SMS
  • Remark 1
  • Theorem 1
  • Remark 2
  • Definition 2
  • Remark 3
  • Remark 4
  • Remark 5
  • Remark 6: Fisher information metric
  • Remark 7