Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PEST: Physics-Enhanced Swin Transformer for 3D Turbulence Simulation

Yilong Dai, Shengyu Chen, Xiaowei Jia, Peyman Givi, Runlong Yu

TL;DR

PEST addresses the core challenges of simulating 3D turbulence by integrating a windowed Swin Transformer with a frequency-adaptive spectral loss and explicit Navier–Stokes/divergence constraints, balanced automatically by uncertainty-based loss weighting. The method enables stable, long-horizon autoregressive rollouts while preserving small-scale structures and respecting physical laws, outperforming nine baselines on two representative turbulence benchmarks. Key innovations include Parseval-based spectral reweighting that preserves locality, and physics-informed penalties that regularize dynamics without destabilizing training. The results demonstrate improved predictive accuracy and physical consistency, suggesting that this architecture can serve as a reliable neural surrogate for complex multi-scale PDEs in fluid dynamics and related domains.

Abstract

Accurate simulation of turbulent flows is fundamental to scientific and engineering applications. Direct numerical simulation (DNS) offers the highest fidelity but is computationally prohibitive, while existing data-driven alternatives struggle with stable long-horizon rollouts, physical consistency, and faithful simulation of small-scale structures. These challenges are particularly acute in three-dimensional (3D) settings, where the cubic growth of spatial degrees of freedom dramatically amplifies computational cost, memory demand, and the difficulty of capturing multi-scale interactions. To address these challenges, we propose a Physics-Enhanced Swin Transformer (PEST) for 3D turbulence simulation. PEST leverages a window-based self-attention mechanism to effectively model localized PDE interactions while maintaining computational efficiency. We introduce a frequency-domain adaptive loss that explicitly emphasizes small-scale structures, enabling more faithful simulation of high-frequency dynamics. To improve physical consistency, we incorporate Navier--Stokes residual constraints and divergence-free regularization directly into the learning objective. Extensive experiments on two representative turbulent flow configurations demonstrate that PEST achieves accurate, physically consistent, and stable autoregressive long-term simulations, outperforming existing data-driven baselines.

PEST: Physics-Enhanced Swin Transformer for 3D Turbulence Simulation

TL;DR

PEST addresses the core challenges of simulating 3D turbulence by integrating a windowed Swin Transformer with a frequency-adaptive spectral loss and explicit Navier–Stokes/divergence constraints, balanced automatically by uncertainty-based loss weighting. The method enables stable, long-horizon autoregressive rollouts while preserving small-scale structures and respecting physical laws, outperforming nine baselines on two representative turbulence benchmarks. Key innovations include Parseval-based spectral reweighting that preserves locality, and physics-informed penalties that regularize dynamics without destabilizing training. The results demonstrate improved predictive accuracy and physical consistency, suggesting that this architecture can serve as a reliable neural surrogate for complex multi-scale PDEs in fluid dynamics and related domains.

Abstract

Accurate simulation of turbulent flows is fundamental to scientific and engineering applications. Direct numerical simulation (DNS) offers the highest fidelity but is computationally prohibitive, while existing data-driven alternatives struggle with stable long-horizon rollouts, physical consistency, and faithful simulation of small-scale structures. These challenges are particularly acute in three-dimensional (3D) settings, where the cubic growth of spatial degrees of freedom dramatically amplifies computational cost, memory demand, and the difficulty of capturing multi-scale interactions. To address these challenges, we propose a Physics-Enhanced Swin Transformer (PEST) for 3D turbulence simulation. PEST leverages a window-based self-attention mechanism to effectively model localized PDE interactions while maintaining computational efficiency. We introduce a frequency-domain adaptive loss that explicitly emphasizes small-scale structures, enabling more faithful simulation of high-frequency dynamics. To improve physical consistency, we incorporate Navier--Stokes residual constraints and divergence-free regularization directly into the learning objective. Extensive experiments on two representative turbulent flow configurations demonstrate that PEST achieves accurate, physically consistent, and stable autoregressive long-term simulations, outperforming existing data-driven baselines.
Paper Structure (74 sections, 1 theorem, 16 equations, 22 figures, 8 tables)

This paper contains 74 sections, 1 theorem, 16 equations, 22 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Formulation
  4. Method
  5. Model Architecture
  6. Swin Transformer Backbone.
  7. Gradient Smoothness Loss.
  8. Frequency-Adaptive Spectral Loss
  9. Physics-Informed Constraints
  10. Divergence Constraint.
  11. Navier-Stokes Residual.
  12. Uncertainty-Based Multi-Loss Balancing
  13. Experiments
  14. Datasets
  15. JHU Isotropic Turbulence.
  16. ...and 59 more sections

Key Result

Theorem 1

For any square-integrable function $f$ defined on a discrete domain of $N$ points and its discrete Fourier transform $\mathcal{F}(f)$, the total energy is preserved:

Figures (22)

  • Figure 1: Overview of the PEST framework. The model takes 5 consecutive flow states as input, processes them through a Swin Transformer encoder--decoder, and predicts the next 5 timesteps. Training combines frequency-adaptive spectral loss, physics constraints, and gradient smoothness, balanced via uncertainty-based weighting.
  • Figure 2: $w$-component predictions on JHU at autoregressive Round 3 ($t=6.0$). Color bar indicates physical-space velocity in m/s. PEST closely matches the DNS solution; baselines show varying degrees of over-smoothing or structural degradation.
  • Figure 3: Per-timestep RMSE and SSIM on JHU across 15 autoregressive steps (3 rounds).
  • Figure 4: Physical consistency analysis on JHU (Round 1). Velocity RMSE vs. four physics metrics; lower-left is better. PEST achieves the best trade-off across all metrics.
  • Figure 5: Spectral ablation: kinetic energy across frequency bands.
  • ...and 17 more figures

Theorems & Definitions (1)

  • Theorem 1: Parseval's Theorem