Implicit factorized transformer approach to fast prediction of turbulent channel flows

TL;DR

This work tackles the challenge of fast, accurate long-term prediction of turbulent channel flows on coarse grids by learning a neural operator. It introduces IFactFormer-m, a modified implicit factorized transformer that employs parallel factorized attention to replace the original chained approach, enabling stable autoregressive predictions. Across Reynolds numbers ${\text{Re}_{\tau} \approx 180, 395, 590}$, IFactFormer-m achieves superior short-term accuracy and maintains high-quality long-term statistics (energy spectrum, mean velocity, rms fluctuations, Reynolds shear stress) compared with IFactFormer-o, FNO, IFNO, and traditional LES (DSM, WALE), while offering substantial speed advantages. The results underscore the potential of transformer-based neural operators as fast, data-driven surrogates for turbulence modeling, with implications for efficient predictive simulations and LES alternatives.

Abstract

Transformer neural operators have recently become an effective approach for surrogate modeling of systems governed by partial differential equations (PDEs). In this paper, we introduce a modified implicit factorized transformer (IFactFormer-m) model which replaces the original chained factorized attention with parallel factorized attention. The IFactFormer-m model successfully performs long-term predictions for turbulent channel flow, whereas the original IFactFormer (IFactFormer-o), Fourier neural operator (FNO), and implicit Fourier neural operator (IFNO) exhibit a poor performance. Turbulent channel flows are simulated by direct numerical simulation using fine grids at friction Reynolds numbers $\text{Re}_τ\approx 180,395,590$, and filtered to coarse grids for training neural operator. The neural operator takes the current flow field as input and predicts the flow field at the next time step, and long-term prediction is achieved in the posterior through an autoregressive approach. The results show that IFactFormer-m, compared to other neural operators and the traditional large eddy simulation (LES) methods including dynamic Smagorinsky model (DSM) and the wall-adapted local eddy-viscosity (WALE) model, reduces prediction errors in the short term, and achieves stable and accurate long-term prediction of various statistical properties and flow structures, including the energy spectrum, mean streamwise velocity, root mean square (rms) values of fluctuating velocities, Reynolds shear stress, and spatial structures of instantaneous velocity. Moreover, the trained IFactFormer-m is much faster than traditional LES methods. By analyzing the attention kernels, we elucidate the reasons why IFactFormer-m converges faster and achieves a stable and accurate long-term prediction compared to IFactFormer-o. Code and data are available at: https://github.com/huiyu-2002/IFactFormer-m.

Figures (14)

• Figure 1: (a) The original factorized attention, which processes each axis sequentially; (b) The modified factorized attention, which processes each axis in parallel.
• Figure 2: Overall design of IFactFormer-m. The left side presents the overall framework based on implicit iteration. The right side illustrates the internal structure of the parallel axial integration layer (PAI-layer).
• Figure 3: The test loss of FNO, IFNO, IFactFormer-o and IFactFormer-m models at various Reynolds numbers: (a) $\text{Re}_{\tau}\approx 180$; (b) $\text{Re}_{\tau}\approx 395$; (c) $\text{Re}_{\tau}\approx 590$. Note that the test loss of different models when untrained is omitted here.
• Figure 4: The correlation coefficient curve of streamwise velocity using FNO, IFNO, IFactFormer-o and IFactFormer-m models at various Reynolds numbers: (a) $\text{Re}_{\tau}\approx 180$; (b) $\text{Re}_{\tau}\approx 395$; (c) $\text{Re}_{\tau}\approx 590$.
• Figure 5: The time series of velocity at position $[2\pi,0.27,2\pi/3]$ at $\text{Re}_{\tau}\approx 590$: (a) streamwise velocity; (b) wall-normal velocity; (c) spanwise velocity.