A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder

TL;DR

A fast and accurate physics-informed neural network ROM, namely nonlinear manifold ROM (NM-ROM), which can better approximate high-fidelity model solutions with a smaller latent space dimension than the LS-ROMs.

Abstract

Traditional linear subspace reduced order models (LS-ROMs) are able to accelerate physical simulations, in which the intrinsic solution space falls into a subspace with a small dimension, i.e., the solution space has a small Kolmogorov n-width. However, for physical phenomena not of this type, e.g., any advection-dominated flow phenomena, such as in traffic flow, atmospheric flows, and air flow over vehicles, a low-dimensional linear subspace poorly approximates the solution. To address cases such as these, we have developed a fast and accurate physics-informed neural network ROM, namely nonlinear manifold ROM (NM-ROM), which can better approximate high-fidelity model solutions with a smaller latent space dimension than the LS-ROMs. Our method takes advantage of the existing numerical methods that are used to solve the corresponding full order models. The efficiency is achieved by developing a hyper-reduction technique in the context of the NM-ROM. Numerical results show that neural networks can learn a more efficient latent space representation on advection-dominated data from 1D and 2D Burgers' equations. A speedup of up to 2.6 for 1D Burgers' and a speedup of 11.7 for 2D Burgers' equations are achieved with an appropriate treatment of the nonlinear terms through a hyper-reduction technique. Finally, a posteriori error bounds for the NM-ROMs are derived that take account of the hyper-reduced operators.

Figures (17)

• Figure 1.1: The figure shows the hierarchy of several ROMs. If the governing equation is nonlinear, then a hyper-reduction is required to achieve both accuracy and speed-up with respect to the corresponding FOM. This paper contributes to the development of NM-LSPG-HR and NM-Galerkin-HR that achieve both speedup and accuracy with the NM-ROM. Throughout the paper, we will compare the performance of LS-ROMs and NM-ROMs.
• Figure 3.1: General description of an autoencoder: $\boldsymbol x$ being encoded to a latent vector, $\hat{\boldsymbol x}$, by the encoder and decoded by the decoder, to $\tilde{\boldsymbol x}$. The mean square error between $\boldsymbol x$ and $\tilde{\boldsymbol x}$ is minimized to update neural network weights and bias.
• Figure 3.2: Three layer autoencoder architecture: (a) unmasked and (b) masked shallow neural neural network. Nodes and edges in orange color represent active path that stems from the sampled outputs that are marked as the orange disks. Note that the masked shallow neural network has a sparser structure than the unmasked one.
• Figure 3.3: Mask matrix. Note that the mask matrices have the analogical structure to the ones of Mass matrix that arises from a numerical discretization, such as the finite element or difference method, with 1D or 2D diffusion equations.
• Figure 3.4: Loss history of decoders for various problems; all three figures show good agreement between train and test loss history, which is a sign for good balance between overfitting and accuracy.