Deep Invertible Autoencoders for Dimensionality Reduction of Dynamical Systems

Abstract

Constructing reduced-order models (ROMs) capable of efficiently predicting the evolution of high-dimensional, parametric systems is crucial in many applications in engineering and applied sciences. A popular class of projection-based ROMs projects the high-dimensional full-order model (FOM) dynamics onto a low-dimensional manifold. These projection-based ROMs approaches often rely on classical model reduction techniques such as proper orthogonal decomposition (POD) or, more recently, on neural network architectures such as autoencoders (AEs). In the case that the ROM is constructed by the POD, one has approximation guaranteed based based on the singular values of the problem at hand. However, POD-based techniques can suffer from slow decay of the singular values in transport- and advection-dominated problems. In contrast to that, AEs allow for better reduction capabilities than the POD, often with the first few modes, but at the price of theoretical considerations. In addition, it is often observed, that AEs exhibits a plateau of the projection error with the increment of the dimension of the trial manifold. In this work, we propose a deep invertible AE architecture, named inv-AE, that improves upon the stagnation of the projection error typical of traditional AE architectures, e.g., convolutional, and the reconstructions quality. Inv-AE is composed of several invertible neural network layers that allows for gradually recovering more information about the FOM solutions the more we increase the dimension of the reduced manifold. Through the application of inv-AE to a parametric 1D Burgers' equation and a parametric 2D fluid flow around an obstacle with variable geometry, we show that (i) inv-AE mitigates the issue of the characteristic plateau of (convolutional) AEs and (ii) inv-AE can be combined with popular projection-based ROM approaches to improve their accuracy.

Figures (9)

• Figure 1: Dimensionality reduction with inv-AE. The snapshots $\bm{x}_k^{\boldsymbol{\mu}}$ are encoded using the encoder $\boldsymbol{\phi}(\bm{x}^{\boldsymbol{\mu}}_k;\boldsymbol{\theta}_{\boldsymbol{\phi}})$, successively zero-masked up to a chosen latent dimension, and eventually decoded using the inverse of the encoder $\boldsymbol{\phi}^{-1}(\bm{z}^{\boldsymbol{\mu}}_k;\boldsymbol{\theta}_{\boldsymbol{\phi}})$.
• Figure 2: Dimensionality reduction with POD-inv-AE. The snapshots dimensionality is first reduced using POD, then the POD coefficients are further compressed by zero-masking output of encoder $\boldsymbol{\phi}(\bm{x}^{\boldsymbol{\mu}}_k;\boldsymbol{\theta}_{\boldsymbol{\phi}})$. The POD coefficients are recovered by decoding the latent variables $\boldsymbol{\phi}^{-1}(\bm{z}^{\boldsymbol{\mu}}_k;\boldsymbol{\theta}_{\boldsymbol{\phi}})$, and eventually the snapshots are reconstructed from the POD coefficients using $U$.
• Figure 3: Invertible affine coupling layer. The forward pass through the layer is indicated by dark khaki arrows, while the backward pass by teal arrows.
• Figure 4: Examples of solution snapshots of the Burgers' equation for the two different instances of the parameter $\boldsymbol{\mu}_{\text{test}}$.
• Figure 5: Projection errors \ref{['eq:proj_error']} calculated on the test set $\bm{X}_{\text{test}}$ for different dimensions of the trial manifold.