Benchmarking Long Roll-outs of Auto-regressive Neural Operators for the Compressible Navier-Stokes Equations with Conserved Quantity Correction

TL;DR

This work tackles the instability of auto-regressive neural operators when simulating PDEs by introducing conserved quantity correction (CQC), a model-agnostic mechanism that enforces mass and momentum conservation during long rollouts of compressible Navier–Stokes flows. By composing a learnable model $\mathcal{G}_\theta$ with a conservation transform $\mathcal{T}$, and applying two correction schemes—Magnitude for nonnegative scalars and Shift for vectors—the authors stabilize predictions across architectures (FNO and DPOT) and improve long-horizon accuracy and correlation, particularly on low-Mach, highly dynamic flows. Spectral analysis reveals that while CQC reduces error growth, current architectures still struggle to capture high-frequency content essential for turbulence, motivating future design focused on high-frequency components. Overall, the approach enhances the reliability of neural operators for turbulent flow prediction and provides a framework to integrate physical conservation into operator learning while highlighting directions to address high-frequency modeling challenges.

Abstract

Deep learning has been proposed as an efficient alternative for the numerical approximation of PDE solutions, offering fast, iterative simulation of PDEs through the approximation of solution operators. However, deep learning solutions have struggle to perform well over long prediction durations due to the accumulation of auto-regressive error, which is compounded by the inability of models to conserve physical quantities. In this work, we present conserved quantity correction, a model-agnostic technique for incorporation physical conservation criteria within deep learning models. Our results demonstrate consistent improvement in the long-term stability of auto-regressive neural operator models, regardless of the model architecture. Furthermore, we analyze the performance of neural operators from the spectral domain, highlighting significant limitations of present architectures. These results highlight the need for future work to consider architectures that place specific emphasis on high frequency components, which are integral to the understanding and modeling of turbulent flows.

Figures (6)

• Figure 1: An example of the modeling framework for the DPOT architecture. Left: the Fourier Attention Layer, comprised of a Fourier transform, multi-head block attention, inverse Fourier transform, and an MLP. Middle: the general model architecture, including patch encoding, Fourier Attention Layers, and patch decoding. Right: the Conserved Quantity Correction Scheme, which calculates the initial quantity and corrects the model output via magnitude or shift scaling. A similar framework is used for FNO.
• Figure 2: A schematic of the FNO architecture, including lifting, spectral convolutions, channel-wise MLPs, and projection operations. This architecture can be used in place of the DPOT architecture in Figure \ref{['ch5.fig:schema']}.
• Figure 3: $L_2$ relative error for models over a long prediction interval. The models with mass and momentum quantity correction (denoted with a $c$ subscript) consistently achieve lower error than their baseline and mass conserving (denoted with a $m$ subscript) counterparts.
• Figure 4: Final timestep predictions for the FNO, FNO$_c$, DPOT, and DPOT$_c$ models for a long rollout test sample. $r$ indicates the correlation between the predicted sample and the ground truth.
• Figure 5: TKE spectra for an example from the Mach 0.1 dataset. FNO-based models are presented in the left plot, while DPOT-based models are presented on the right. While both models exhibit lower density than the ground-truth for high frequency modes, the DPOT models show a lesser difference than the FNO models.