Data-Augmented Predictive Deep Neural Network: Enhancing the extrapolation capabilities of non-intrusive surrogate models

TL;DR

A new deep learning framework is proposed, where kernel dynamic mode decomposition (KDMD) is employed to evolve the dynamics of the latent space generated by the encoder part of a convolutional autoencoder (CAE) to improve the extrapolation capabilities of the surrogate models in the entire time domain.

Abstract

Numerically solving a large parametric nonlinear dynamical system is challenging due to its high complexity and the high computational costs. In recent years, machine-learning-aided surrogates are being actively researched. However, many methods fail in accurately generalizing in the entire time interval $[0, T]$, when the training data is available only in a training time interval $[0, T_0]$, with $T_0<T$. To improve the extrapolation capabilities of the surrogate models in the entire time domain, we propose a new deep learning framework, where kernel dynamic mode decomposition (KDMD) is employed to evolve the dynamics of the latent space generated by the encoder part of a convolutional autoencoder (CAE). After adding the KDMD-decoder-extrapolated data into the original data set, we train the CAE along with a feed-forward deep neural network using the augmented data. The trained network can predict future states outside the training time interval at any out-of-training parameter samples. The proposed method is tested on two numerical examples: a FitzHugh-Nagumo model and a model of incompressible flow past a cylinder. Numerical results show accurate and fast prediction performance in both the time and the parameter domain.

Figures (10)

• Figure 1: AE: AE generates a latent space with a nonlinear mapping.
• Figure 2: CAE-FFNN: CAE is responsible for the data compression and the data reconstruction. The FFNN connects the parameter-time space to the latent space.
• Figure 3: CAE-KDMD: In the latent space generated by CAE, the latent vector at any future time is extrapolated by KDMD and then decoded into the states in the physical space.
• Figure 4: Data-augmented Predictive DNN flowchart: Lower-left part, pretraining of the CAE. Upper-left part, the generation of the complementary data. On the right side, train CAE-FFNN with the augmented data at the offline stage, and prediction at new parameter-time pairs with the trained FFNN-decoder in the online phase.
• Figure 5: FitzHugh-Nagumo model: the predicted solution and the reference solution. (a) The outputs $v(0,\varepsilon^*,t)$ and $w(0,\varepsilon^*,t)$ when $\varepsilon^* = 0.0151$. (b) Limit cycles of $v(x,\varepsilon^*,t)$ w.r.t. $w(x,\varepsilon^*,t)$ when $\varepsilon^* = 0.0151$. (c) The outputs $v(0,\varepsilon^*,t)$ and $w(0,\varepsilon^*,t)$ when $\varepsilon^* = 0.0352$. (d) Limit cycles of $v(x,\varepsilon^*,t)$ w.r.t. $w(x,\varepsilon^*,t)$ when $\varepsilon^* = 0.0352$.