Generalization capabilities and robustness of hybrid models grounded in physics compared to purely deep learning models

TL;DR

The paper addresses efficient forecasting of high-dimensional fluid flow evolution by comparing a physics-informed hybrid POD-DL model with two purely deep-learning autoregressive models (Residual Convolutional Autoencoder with ConvLSTM and VAE with ConvLSTM). By projecting high-dimensional data onto a reduced POD space and forecasting in that space, POD-DL achieves superior generalization and robustness for both laminar (3D, $Re=280$) and turbulent (2D, $Re=4000$) cylinder wakes. The results show that POD-DL outperforms the purely DL models in accuracy and stability, with VAE underperforming in turbulence due to the complexity of its ELBO loss. The findings advocate physics-informed hybrid approaches for reduced-order modeling of complex flows and suggest avenues for extending these methods with diffusion-based temporal models and transfer learning to further enhance applicability and efficiency.

Abstract

This study investigates the generalization capabilities and robustness of purely deep learning (DL) models and hybrid models based on physical principles in fluid dynamics applications, specifically focusing on iteratively forecasting the temporal evolution of flow dynamics. Three autoregressive models were compared: a hybrid model (POD-DL) that combines proper orthogonal decomposition (POD) with a long-short term memory (LSTM) layer, a convolutional autoencoder combined with a convolutional LSTM (ConvLSTM) layer and a variational autoencoder (VAE) combined with a ConvLSTM layer. These models were tested on two high-dimensional, nonlinear datasets representing the velocity field of flow past a circular cylinder in both laminar and turbulent regimes. The study used latent dimension methods, enabling a bijective reduction of high-dimensional dynamics into a lower-order space to facilitate future predictions. While the VAE and ConvLSTM models accurately predicted laminar flow, the hybrid POD-DL model outperformed the others across both laminar and turbulent flow regimes. This success is attributed to the model's ability to incorporate modal decomposition, reducing the dimensionality of the data, by a non-parametric method, and simplifying the forecasting component. By leveraging POD, the model not only gained insight into the underlying physics, improving prediction accuracy with less training data, but also reduce the number of trainable parameters as POD is non-parametric. The findings emphasize the potential of hybrid models, particularly those integrating modal decomposition and deep learning, in predicting complex flow dynamics.

Figures (26)

• Figure 1: Overview of the architecture of the POD-DL model. Note, it can be represented as an encoder-decoder structure. However, in this case the encoder is just the matrix decomposition ($\bm U, \bm{\Sigma}, \bm V^{T}$) from SVD. The temporal information is formed by a matrix multiplication $\bm M_{t-n+1}^{t} = \bm{\Sigma} * (\bm V_{t-n+1}^{t})^{T}$, which is send to a DL model, usually a long-short term memory (LSTM), to predict the time-ahead sample, i.e., $\bm M_{t+1}$. Finally, the decoder reconstructs the snapshot by multiplying matrices $\bm x_{t+1} = \bm U * \bm M_{t+1}$.
• Figure 2: Overview of the architecture of the residual autoencoder. Unlike the POD-DL, the encoder and decoder of this model are composed of convolutional neural networks.
• Figure 3: Overview of the architecture of the VAE. Similar to the residual autoencoder, the encoder and decoder of this model are composed of convolutional neural networks.
• Figure 4: Visual representation of both types of blocks, identity (top) and convolutional (bottom). Both of them add up the input and output values of the block. However the identity block (top) does not reduce the spatial dimension of the input snapshot while the convolutional block (bottom) does it.
• Figure 5: Comparison of the ground truth streamwise velocity evolution $u_{x}(t)$, at spatial coordinate $P_{1}$ shown in Fig. \ref{['fig: synthetic_ref_points']}, against the predictions obtained from the (a) POD-DL, (b) residual autoencoder and (c) variational autoencoder models, respectively. Counter part for the spatial coordinate $P_{2}$ in figures (d), (e) and (f). In all figures the ground truth is represented by a solid line and the prediction by a dashed line.