Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Parametric Taylor series based latent dynamics identification neural networks

Xinlei Lin, Dunhui Xiao

TL;DR

A new parametric latent identification of nonlinear dynamics neural networks, P-TLDINets, is introduced, which relies on a novel neural network structure based on Taylor series expansion and ResNets to learn the ODEs that govern the reduced space dynamics.

Abstract

Numerical solving parameterised partial differential equations (P-PDEs) is highly practical yet computationally expensive, driving the development of reduced-order models (ROMs). Recently, methods that combine latent space identification techniques with deep learning algorithms (e.g., autoencoders) have shown great potential in describing the dynamical system in the lower dimensional latent space, for example, LaSDI, gLaSDI and GPLaSDI. In this paper, a new parametric latent identification of nonlinear dynamics neural networks, P-TLDINets, is introduced, which relies on a novel neural network structure based on Taylor series expansion and ResNets to learn the ODEs that govern the reduced space dynamics. During the training process, Taylor series-based Latent Dynamic Neural Networks (TLDNets) and identified equations are trained simultaneously to generate a smoother latent space. In order to facilitate the parameterised study, a $k$-nearest neighbours (KNN) method based on an inverse distance weighting (IDW) interpolation scheme is introduced to predict the identified ODE coefficients using local information. Compared to other latent dynamics identification methods based on autoencoders, P-TLDINets remain the interpretability of the model. Additionally, it circumvents the building of explicit autoencoders, avoids dependency on specific grids, and features a more lightweight structure, which is easy to train with high generalisation capability and accuracy. Also, it is capable of using different scales of meshes. P-TLDINets improve training speeds nearly hundred times compared to GPLaSDI and gLaSDI, maintaining an $L_2$ error below $2\%$ compared to high-fidelity models.

Parametric Taylor series based latent dynamics identification neural networks

TL;DR

A new parametric latent identification of nonlinear dynamics neural networks, P-TLDINets, is introduced, which relies on a novel neural network structure based on Taylor series expansion and ResNets to learn the ODEs that govern the reduced space dynamics.

Abstract

Numerical solving parameterised partial differential equations (P-PDEs) is highly practical yet computationally expensive, driving the development of reduced-order models (ROMs). Recently, methods that combine latent space identification techniques with deep learning algorithms (e.g., autoencoders) have shown great potential in describing the dynamical system in the lower dimensional latent space, for example, LaSDI, gLaSDI and GPLaSDI. In this paper, a new parametric latent identification of nonlinear dynamics neural networks, P-TLDINets, is introduced, which relies on a novel neural network structure based on Taylor series expansion and ResNets to learn the ODEs that govern the reduced space dynamics. During the training process, Taylor series-based Latent Dynamic Neural Networks (TLDNets) and identified equations are trained simultaneously to generate a smoother latent space. In order to facilitate the parameterised study, a -nearest neighbours (KNN) method based on an inverse distance weighting (IDW) interpolation scheme is introduced to predict the identified ODE coefficients using local information. Compared to other latent dynamics identification methods based on autoencoders, P-TLDINets remain the interpretability of the model. Additionally, it circumvents the building of explicit autoencoders, avoids dependency on specific grids, and features a more lightweight structure, which is easy to train with high generalisation capability and accuracy. Also, it is capable of using different scales of meshes. P-TLDINets improve training speeds nearly hundred times compared to GPLaSDI and gLaSDI, maintaining an error below compared to high-fidelity models.
Paper Structure (23 sections, 32 equations, 17 figures, 4 tables, 3 algorithms)

This paper contains 23 sections, 32 equations, 17 figures, 4 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Governing equations of parameterised dynamic systems
  3. Taylor series-based Latent Dynamic Neural Networks (TLDNets)
  4. The dynamic model and the reconstruction model
  5. The initial mapping model
  6. Residual Learning neural Network (ResNet)
  7. Identification of dynamics (ID model) in the latent space
  8. $k$-Nearest Neighbour interpolation with inverse distance weighting
  9. Data pre-processing
  10. Data sampling
  11. Normalisation
  12. Framework of P-TLDINets
  13. Off-line model training
  14. Online stage with the reconstruction model and ID models
  15. Numerical Results
  16. ...and 8 more sections

Figures (17)

  • Figure 1: Residual learning neural network block structure
  • Figure 2: Flowchart of the P-TLDINets algorithm.
  • Figure 3: Online stage of the P-TLDINets algorithm.
  • Figure 4: 2D-Burgers: The inner small blue square represents the range of the small parameter set; the outer large green square represents the range of a large parameter set (includes the blue square area).
  • Figure 5: 2D-burgers: Low-dimensional dynamics and the solutions of the identified equations in the small dataset $\mathcal{D}_1$. (a) and (b) show the dimension of the latent space $N_s$ that ranges from $2$ to $7$. (a) represents the results for the entire time frame $t \in [0, 1.0]$, while (b) shows only in $t \in [0.4, 0.6]$.
  • ...and 12 more figures

Theorems & Definitions (1)

  • Remark 1