EddyFormer: Accelerated Neural Simulations of Three-Dimensional Turbulence at Scale

TL;DR

EddyFormer introduces a two-stream, SEM-tokenized Transformer architecture for large-scale turbulence simulations, combining SGS-local modeling with LES-global attention to capture multi-scale dynamics. By operating on SEM tokens with SEMConv and SEMAttn, and using rotary position encoding, it achieves DNS-level accuracy at $256^3$ for 3D isotropic turbulence while delivering substantial speedups on GPUs. The method generalizes across larger domains and diverse flow conditions, performing well on The Well benchmark where many baselines fail to converge. This approach promises near real-time, high-fidelity turbulence simulations with broad applicability in engineering and geophysical contexts.

Abstract

Computationally resolving turbulence remains a central challenge in fluid dynamics due to its multi-scale interactions. Fully resolving large-scale turbulence through direct numerical simulation (DNS) is computationally prohibitive, motivating data-driven machine learning alternatives. In this work, we propose EddyFormer, a Transformer-based spectral-element (SEM) architecture for large-scale turbulence simulation that combines the accuracy of spectral methods with the scalability of the attention mechanism. We introduce an SEM tokenization that decomposes the flow into grid-scale and subgrid-scale components, enabling capture of both local and global features. We create a new three-dimensional isotropic turbulence dataset and train EddyFormer to achieves DNS-level accuracy at 256^3 resolution, providing a 30x speedup over DNS. When applied to unseen domains up to 4x larger than in training, EddyFormer preserves accuracy on physics-invariant metrics-energy spectra, correlation functions, and structure functions-showing domain generalization. On The Well benchmark suite of diverse turbulent flows, EddyFormer resolves cases where prior ML models fail to converge, accurately reproducing complex dynamics across a wide range of physical conditions.

Figures (16)

• Figure 1: Vorticity magnitude of forced turbulence with Reynolds no. $Re \approx 94$ at $t = 5$. EddyFormer reproduces the accuracy level and invariant flow statistics of a direct numerical simulation (DNS) at $256^3$ resolution, computed with highly efficient pseudo-spectral schemes canuto2007spectral. The reported L2 error is measured against a reference DNS at $384^3$ resolution. When benchmarked on a single NVIDIA A100 GPU, EddyFormer is $30\mathsf{x}$ faster, achieving essentially real-time simulations.
• Figure 2: Our proposed architecture: EddyFormer. (a) The PDE initial conditions are interpolated using spectral element methods (SEM). The large-eddy simulation (LES) field and subgrid-scale (SGS) field are transformed through the LES stream (§\ref{['sec:les-stream']}) and SGS stream (§\ref{['sec:sgs-stream']}), respectively. The SGS stream consists of (b) SEM-based convolution blocks to model local eddy dynamics, while the LES stream consists of (c) SEM-based attention blocks to capture long-range dependencies.
• Figure 3: Performance of models and DNS on Re94 (§\ref{['exp:3d']}), a 3D homogeneous isotropic turbulence fluid flow. The models learn to correct DNS at $96^3$ resolution, and we report evaluation metrics relative to a reference DNS at $384^3$ resolution. EddyFormer achieves the accuracy of DNS at $256^3$ resolution in terms of (a) relative L2 error for $t \leq 5$ and (b) correlation for $5 \leq t \leq 10$ which measures field similarities; for (c) longer prediction rollouts up to $20$ seconds, EddyFormer also captures the third-order structure function, Eqn. (\ref{['eqn:s3']}), which measures the velocity skewness and indicates the energy transfer across different scales. In contrast, the baseline F-FNO shows higher rollout error and a mismatched structure function.
• Figure 4: Visualization of the Q-criterion for a Re94 (§\ref{['exp:3d']}) test sample at $t = 5$. The Q-criterion, Eqn. (\ref{['eqn:q']}), identifies vortex structures where rotational motion dominates over strain, with the red volume in (a) reference DNS solution highlighting the vortex cores; (b) EddyFormer successfully captures both vortex cores, while (c) F-FNO fails to resolve the vortex at the center of the domain.
• Figure 5: (Left) Illustration of how EddyFormer generalizes to larger domains. Trained on a smaller domain, EddyFormer predicts the flow field on a larger domain by applying an attention mask that ensures consistency with the training setup. The context window of each SEM token is constrained to a square of length $2\pi$, maintaining consistency between training and test times. (Right) Model evaluation on both $2\mathsf{x}$ and $4\mathsf{x}$ larger domains of KF4 (§\ref{['exp:2d']}). The scaled energy spectra, Eqn. (\ref{['eqn:ek']}), of the EddyFormer's predictions align closely with the reference high-resolution DNS, while F-FNO does not capture this and shows deviation in high-frequency modes.