MFC 5.0: An exascale many-physics flow solver

TL;DR

MFC 5.0 presents an exascale, multiphysics CFD framework that unifies diffuse-interface multiphase modeling (five- and six-equation formulations) with phase change, bubble dynamics, FSI, chemistry, surface tension, and MHD/RMHD. It couples advanced numerics (WENO-Z/TENO, general characteristic boundaries, Strang splitting, low-Mach treatment) with portable GPU/APU offloading and code-generation for thermochemistry, backed by rigorous CI and extreme-scale I/O. The work demonstrates state-of-the-art performance and scalability on Frontier and El Capitan, including a 200 trillion-grid-point simulation and Gordon Bell Prize finalist status, underscoring the practical viability of end-to-end, high-fidelity simulations across engineering, medicine, and science. Collectively, these contributions position MFC 5.0 as a versatile, portable, and scalable platform for credible, high-fidelity simulations of complex multiphysics flows at exascale.

Abstract

Many problems of interest in engineering, medicine, and the fundamental sciences rely on high-fidelity flow simulation, making performant computational fluid dynamics solvers a mainstay of the open-source software community. Previous work, MFC 3.0, was published, documented, and made open-source by Bryngelson et al. CPC (2021) features numerous physical features, numerical methods, and scalable infrastructure. MFC 5.0 is a significant update to MFC 3.0, featuring a broad set of well-established and novel physical models and numerical methods, as well as the introduction of GPU and APU (or superchip) acceleration. We exhibit state-of-the-art performance and ideal scaling on the first two exascale supercomputers, OLCF's Frontier and LLNL's El Capitan. Combined with MFC's single-accelerator performance, MFC achieves exascale computation in practice and has achieved the largest-to-date public CFD simulation at 200 trillion grid points, earning it a 2025 ACM Gordon Bell Prize finalist. New physical features include the immersed boundary method, $N$-fluid phase change, Euler-Euler and Euler-Lagrange sub-grid bubble models, fluid-structure interaction, hypo- and hyper-elastic materials, chemically reacting flow, two-material surface tension, magnetohydrodynamics (MHD), and more. Numerical techniques now represent the current state-of-the-art, including general relaxation characteristic boundary conditions, WENO variants, Strang splitting for stiff sub-grid flow features, and low Mach number treatments. Weak scaling to tens of thousands of GPUs on OLCF's Summit and Frontier, and LLNL's El Capitan, achieves efficiencies within 5% of ideal to over 90% of their respective system sizes. Strong scaling results for a 16-fold increase in device count show parallel efficiencies exceeding 90% on OLCF Frontier.

Figures (22)

• Figure 1: Convergence results for a 1D 2-component advection problem with the (a) HLL and (b) HLLC approximate Riemann solvers.
• Figure 2: Steady density field ($\rho / \rho_0$) of Mach 2 helium flow over a sphere with (a) analytical level set and (b) STL-based level set.
• Figure 3: Comparison between the numerical and experimental values for the evaporation front velocity using the 5- and 6-equation models for the chocked flow series of n-dodecane tests simoesmoreira1999evaporation.
• Figure 4: Volume spreading of the $n$-th bubble for the void fraction computations using a Gaussian kernel of characteristic radial extent (not to scale).
• Figure 5: Evolution of an isolated bubble in response to a single cycle of a sinusoidal pressure wave. The analytical solution of the Keller--Miksis equation was reported by maeda2018eulerian.