Physics-Informed Transformer operator for the prediction of three-dimensional turbulence

TL;DR

This work tackles data dependency and interpretability in 3D turbulence prediction by introducing physics-informed Transformer operators PITO and PIITO, built on a patch-based Vision Transformer (ViTO) to map the current 3D flow field to its next state. The models embed LES equations and an SGS closure into a PDE loss, enabling data-free learning and even automatic learning of the SGS coefficient from a single flow dataset. Across decaying and forced HIT, PITO/PIITO outperform the physics-informed Fourier neural operator (PIFNO), achieving long-term stability beyond 25× the training horizon and substantial memory and parameter reductions while preserving or improving accuracy. The approach offers a scalable, data-efficient path for 3D turbulence prediction and can be extended to more complex geometries and SGS models.

Abstract

Data-driven turbulence prediction methods often face challenges related to data dependency and lack of physical interpretability. In this paper, we propose a physics-informed Transformer operator (PITO) and its implicit variant (PIITO) for predicting three-dimensional (3D) turbulence, which are developed based on the vision Transformer (ViT) architecture with an appropriate patch size. Given the current flow field, the Transformer operator computes its prediction for the next time step. By embedding the large-eddy simulation (LES) equations into the loss function, PITO and PIITO can learn solution operators without using labeled data. Furthermore, PITO can automatically learn the subgrid scale (SGS) coefficient using a single set of flow data during training. Both PITO and PIITO exhibit excellent stability and accuracy on the predictions of various statistical properties and flow structures for the situation of long-term extrapolation exceeding 25 times the training horizon in decaying homogeneous isotropic turbulence (HIT), and outperform the physics-informed Fourier neural operator (PIFNO). Furthermore, PITO exhibits a remarkable accuracy on the predictions of forced HIT where PIFNO fails. Notably, PITO and PIITO reduce GPU memory consumption by 79.5\% and 91.3\% while requiring only 31.5\% and 3.1\% of the parameters, respectively, compared to PIFNO. Moreover, both PITO and PIITO models are much faster compared to traditional LES method.

Figures (22)

• Figure 1: (a) The architecture of the ViTO; (b) The Multi-Head attention mechanism in the Transformer block.
• Figure 2: The architecture of the IViTO
• Figure 3: The architecture of the PITO and PIITO frameworks constrained by LES equations.
• Figure 4: Evolutions of loss curves in decaying HIT at the stationary initial condition: (a) PIFNO; (b) PITO; (c) PIITO and (d) comparison of test loss during training.
• Figure 5: Temporal evolutions of (a) the root-mean-square (rms) velocity and (b) the rms of vorticity for different models in decaying HIT at the stationary initial condition.