An Efficient Graph-Transformer Operator for Learning Physical Dynamics with Manifolds Embedding

TL;DR

PhysGTO introduces an efficient Graph-Transformer Operator that embeds manifold structure in both physical and latent spaces to learn physical dynamics on unstructured meshes. It combines a topology-aware, flux-oriented message-passing mechanism with a projection-inspired, linear-complexity attention to capture local and global dependencies, achieving linear complexity in mesh size. A topology-aware mesh sampler enables scalable training on large meshes while maintaining accuracy, and a learnable subspace attention basis allows robust long-range reasoning. Across eleven datasets spanning static and transient 3D geometries, PhysGTO achieves state-of-the-art accuracy with substantial reductions in parameters and FLOPs, demonstrating both high fidelity and practical efficiency for real-world physics surrogates.

Abstract

Accurate and efficient physical simulations are essential in science and engineering, yet traditional numerical solvers face significant challenges in computational cost when handling simulations across dynamic scenarios involving complex geometries, varying boundary/initial conditions, and diverse physical parameters. While deep learning offers promising alternatives, existing methods often struggle with flexibility and generalization, particularly on unstructured meshes, which significantly limits their practical applicability. To address these challenges, we propose PhysGTO, an efficient Graph-Transformer Operator for learning physical dynamics through explicit manifold embeddings in both physical and latent spaces. In the physical space, the proposed Unified Graph Embedding module aligns node-level conditions and constructs sparse yet structure-preserving graph connectivity to process heterogeneous inputs. In the latent space, PhysGTO integrates a lightweight flux-oriented message-passing scheme with projection-inspired attention to capture local and global dependencies, facilitating multilevel interactions among complex physical correlations. This design ensures linear complexity relative to the number of mesh points, reducing both the number of trainable parameters and computational costs in terms of floating-point operations (FLOPs), and thereby allowing efficient inference in real-time applications. We introduce a comprehensive benchmark spanning eleven datasets, covering problems with unstructured meshes, transient flow dynamics, and large-scale 3D geometries. PhysGTO consistently achieves state-of-the-art accuracy while significantly reducing computational costs, demonstrating superior flexibility, scalability, and generalization in a wide range of simulation tasks.

Figures (16)

• Figure 1: The overall pipeline of PhysGTO, an efficient Graph-Transformer Operator for learning physical dynamics. (a) Taking multiple conditions as inputs, PhysGTO can predict velocity fields for temporal problems and estimate aerodynamic coefficients for forward problems. (b) Differences between base blocks, including CNNs, GNNs, Transformers, and our proposed GTO. GTO integrates local and global interactions enabling effective information flow across unstructured domains. (c) Detailed structure of the Unified Graph Embedding (UGE) module, which encodes diverse input conditions into a unified graph representation. UGE consists of two components: Multi-Condition Aligner, and Topology-Aware Mesh Sampler. (d) Internal architecture of a GTO block. Each block includes two core components: Flux-Oriented Message Passing for local interactions, and Projection-Inspired Attention for long-range dependencies.
• Figure 1: Visualization of additional representative cases comparing PhysGTO with the SOTA NORM for Darcy problem. Here, Error refers to $|\hat{u}(x,y) - u(x,y)|$, where $\hat{u}(x,y)$ and $u(x,y)$ denote the predicted and ground-truth values, respectively.
• Figure 2: PhysGTO excels in learning complex physical patterns on unstructured meshes: a comparison with a state-of-the-art model, NORM chen2024learning. (a) Raincloud plots depicting the relative $L_2$ errors of the current SOTA model and the proposed PhysGTO across five datasets. (b-d) Visualization of some representative cases: (b) Darcy problem, (c) Heat transfer, and (d) Blood flow. Here, Abs Error refers to $|\hat{u}(x) - u(x)|$, where $\hat{u}(x)$ and $u(x)$ denote the predicted and ground-truth values, respectively. More visualization results can be found in SI Sec. \ref{['supp visulaization']}.
• Figure 2: Visualization of additional representative cases comparing PhysGTO with the SOTA NORM for the prediction in the magnitude of velocity $u(x,y)$ in Pipe Turbulence problem. Here, Error refers to $|\hat{u}(x,y) - u(x,y)|$, where $\hat{u}(x,y)$ and $u(x,y)$ denote the predicted and ground-truth values, respectively.
• Figure 3: PhysGTO enables reliable dynamics forecasting over long time horizons. (a) Raincloud plots depicting the relative $L_2$ errors of SOTA models and the proposed PhysGTO across four datasets. (b) Relative $L_2$ errors at each time step over long temporal horizons across four datasets, comparing four deep learning models. (c-d) Visualization of two representative cases: (c) Cylinder Flow, (d) EAGLE dataset. The first row shows the ground-truth velocity magnitude ($\sqrt{u^2+v^2}$), while the second and third rows present the prediction errors of the state-of-the-art baseline and the proposed PhysGTO, respectively. Here, Abs Error refers to $|\hat{u}(x) - u(x)|$, where $\hat{u}(x)$ and $u(x)$ denote the predicted and ground-truth values, respectively.