FLUID-LLM: Learning Computational Fluid Dynamics with Spatiotemporal-aware Large Language Models

TL;DR

FLUID-LLM tackles the computational burden of CFD by marrying pre-trained large language models with spatiotemporal encodings to forecast unsteady fluid flows. The framework tokenizes and embeds patch-based fluid states, augments them with learned spatiotemporal context, and uses an LLM to produce next-state embeddings that a grid-based GNN decoder converts into state updates via $\hat{s}_{t+1} = s_t + f_{\text{dec}}(e^{\text{out}}_t)$. Across Cylinder and Airfoil benchmarks, FLUID-LLM with larger LLMs achieves superior long-horizon accuracy, and experiments demonstrate in-context learning and few-shot adaptation to novel dynamics. This indicates a practical path toward efficient CFD with language-model priors and flexible spatiotemporal reasoning.

Abstract

Learning computational fluid dynamics (CFD) traditionally relies on computationally intensive simulations of the Navier-Stokes equations. Recently, large language models (LLMs) have shown remarkable pattern recognition and reasoning abilities in natural language processing (NLP) and computer vision (CV). However, these models struggle with the complex geometries inherent in fluid dynamics. We introduce FLUID-LLM, a novel framework combining pre-trained LLMs with spatiotemporal-aware encoding to predict unsteady fluid dynamics. Our approach leverages the temporal autoregressive abilities of LLMs alongside spatial-aware layers, bridging the gap between previous CFD prediction methods. Evaluations on standard benchmarks reveal significant performance improvements across various fluid datasets. Our results demonstrate that FLUID-LLM effectively integrates spatiotemporal information into pre-trained LLMs, enhancing CFD task performance.

Figures (9)

• Figure 1: High-level overview of the FLUID-LLM framework. Given an input context history of fluid states, we first tokenize (1) and embed it via patch encoding (2) along with a spatiotemporal embedding (3). These combined embeddings are then given as input to a fine-tuned language model (LLM). Then, the LLM produces updated embeddings by forwarding them through its internal layers (4). These are projected into a grid, updating each state embedding with a GNN to allow for information propagation between states in the grid (5). Finally, the network predicts the differences between the previous and next state to generate the subsequent fluid state.
• Figure 2: Predicted RMSE after 50 steps on the Airfoil dataset, where the model is given $\tau_\text{max}$ context states.
• Figure 3: Predicted RMSE after 1 and 10 steps on the synthetic dataset, where the model is given $\tau_\text{init}$ initial context states.
• Figure A4: Plots of predictions for our models and baselines on the Cylinder dataset for the $V_x$ component, at different numbers of prediction steps. This particular sample involves simulating vortex instabilities for an incompressible fluid flow in a tube.
• Figure A5: Plots of predictions for our models and baselines on the Airfoil dataset for the $V_x$ component, at different numbers of prediction steps.