Predicting Physics in Mesh-reduced Space with Temporal Attention

TL;DR

Next-step GNNs on irregular meshes suffer from error accumulation and drift during long rollouts. The authors propose a mesh-reduced, transformer-based autoregressive simulator that operates in a low-dimensional latent space via GMR/GMUS, enabling long-horizon predictions with temporal attention over latent sequences. The approach outperforms MeshGraphNet across multiple fluid dynamics tasks, achieves stable rollouts without training noise, and provides memory and compute benefits through mesh reduction. This framework enables efficient, phase-stable surrogates for high-dimensional physics on irregular meshes, with potential benefits for real-time simulation and optimization.

Abstract

Graph-based next-step prediction models have recently been very successful in modeling complex high-dimensional physical systems on irregular meshes. However, due to their short temporal attention span, these models suffer from error accumulation and drift. In this paper, we propose a new method that captures long-term dependencies through a transformer-style temporal attention model. We introduce an encoder-decoder structure to summarize features and create a compact mesh representation of the system state, to allow the temporal model to operate on a low-dimensional mesh representations in a memory efficient manner. Our method outperforms a competitive GNN baseline on several complex fluid dynamics prediction tasks, from sonic shocks to vascular flow. We demonstrate stable rollouts without the need for training noise and show perfectly phase-stable predictions even for very long sequences. More broadly, we believe our approach paves the way to bringing the benefits of attention-based sequence models to solving high-dimensional complex physics tasks.

Figures (18)

• Figure 1: The diagram of the proposed model, GMR-Transformer-GMUS. We first represent the domain as a graph and then select pivotal nodes (red/green/yellow) to encode information over the entire graph. The encoder GMR runs Message passing along graph edges so that the pivotal nodes collect information from nearby nodes. The latent vector $\bm{z}_t$ summarizes information at the pivotal nodes, and represents the whole domain at the current step. The transformer will predict $\bm{z}_{t+1}$ based on all previous state latent vectors. Finally, we decode $\bm{z}_{t+1}$ through message passing to obtain the next-step prediction $\mathbf{Y}_{t+1}$.
• Figure 2: Contours of the velocity field, as predicted by our model versus the ground truth (CFD). Our model accurately predicts long rollout sequences under varying system parameters.
• Figure 3: Averaged error over all state variables on cylinder flow (left), sonic flow (middle) and vascular flow (right), for the models MeshGraphNets(MGN)(\ref{['line:case0:GMNpure']}), MGN-NI(\ref{['line:case0:GMN_noiseInjection']}), Ours-GRU (\ref{['line:case0:GMRGRU']}), Ours-LSTM (\ref{['line:case0:GMRLSTM']}), Ours-Transformer (\ref{['line:case0:ourModal']}). Our model, particularly the transformer, show much less error accumulation compared to the next-step model.
• Figure 4: Predictions of the next-step MeshGraphNet model and our model, compared to ground truth. The next-step model fails to keep the shedding frequency and show drifts on cylinder flow. On vascular flow, we notice the left inflow diminishing over the time, while our model remains close to the reference simulation.
• Figure 5: 2-D principle subspace of the latent vectors from GMR (left) and PCA (middle) for flow past cylinder system: $Re = 307$ (\ref{['ls:LowReTraj']}), $Re = 993$ (\ref{['ls:HighReTraj']}) start from \ref{['ls:LowReTrajStart']} and \ref{['ls:HighReTrajStart']} respectively.