Towards Multi-Fidelity Scaling Laws of Neural Surrogates in CFD

TL;DR

The paper addresses the data-cost challenge in scientific ML by introducing multi-fidelity scaling laws that split the data axis into a compute budget $D_b$ and fidelity composition $D_c$. It evaluates a transformer-based neural surrogate on a CFD Airfoil dataset with paired low- and high-fidelity simulations to reveal a compute-budget scaling law and budget-dependent optimal fidelity mixes. A physical explanation links transfer patterns to boundary-layer treatment, showing positive LF-to-HF transfer for some fields but not for wall shear stress due to near-wall modeling differences. The findings offer practical guidance for compute-efficient dataset generation in scientific ML and outline directions for extending fidelities, adopting continuous fidelities, and generalizing to other domains.

Abstract

Scaling laws describe how model performance grows with data, parameters and compute. While large datasets can usually be collected at relatively low cost in domains such as language or vision, scientific machine learning is often limited by the high expense of generating training data through numerical simulations. However, by adjusting modeling assumptions and approximations, simulation fidelity can be traded for computational cost, an aspect absent in other domains. We investigate this trade-off between data fidelity and cost in neural surrogates using low- and high-fidelity Reynolds-Averaged Navier-Stokes (RANS) simulations. Reformulating classical scaling laws, we decompose the dataset axis into compute budget and dataset composition. Our experiments reveal compute-performance scaling behavior and exhibit budget-dependent optimal fidelity mixes for the given dataset configuration. These findings provide the first study of empirical scaling laws for multi-fidelity neural surrogate datasets and offer practical considerations for compute-efficient dataset generation in scientific machine learning.

Figures (3)

• Figure 1: Visual comparison of a low- and high-fidelity simulation of a NACA4 airfoil with parameters (M=2.408, P=5.987, XX=11.876), at an AoA of 7.57° with an inlet velocity of 81.645ms. \ref{['fig:af_sub1']} (left) shows the $x$-velocity of the high-fidelity simulation, and (right) zooms on the corresponding boundary layer profile at mid-chord ($0.5c$), highlighting the difference in modeling resolution between low- and high-fidelity. Bottom plots display the evolution of the pressure coefficient $C_p$ (\ref{['fig:af_sub2']}) and skin friction coefficient $C_{\tau}$ (\ref{['fig:af_sub3']}), along the chord line $x/c$.
• Figure 2: Scaling behavior for fields with positive transfer. We show the MSE of normalized fields averaged over four seeds with error bars indicating standard deviation. Lines of different colors show different training budgets in compute hours $D_b$, at growing percentage of high-fidelity composition $D_c$. The dashed line indicates model performance when trained on the full high-fidelity dataset.
• Figure 3: Scaling behavior for fields without positive transfer. We show the MSE of normalized fields averaged over four seeds with error bars indicating standard deviation. Lines of different colors show different training budgets in compute hours $D_b$, at growing percentage of high-fidelity composition $D_c$. The dashed line indicates model performance when trained on the full high-fidelity dataset.