PGOT: A Physics-Geometry Operator Transformer for Complex PDEs

TL;DR

PGOT tackles the challenge of modeling PDEs on large, unstructured meshes by addressing geometric aliasing that arises in efficient linear-transformer approaches. It introduces Spectrum-Preserving Geometric Attention to inject multi-scale geometry alongside a Taylor-decomposed FFN to adaptively route computations between linear and non-linear dynamics. The combination yields state-of-the-art performance on standard PDE benchmarks and industrial simulations, with strong out-of-distribution robustness and scalable $O(N)$ complexity. This geometry-physics decoupling enhances boundary fidelity and design-optimization reliability, offering a practical path toward high-fidelity, real-time scientific computing on complex geometries.

Abstract

While Transformers have demonstrated remarkable potential in modeling Partial Differential Equations (PDEs), modeling large-scale unstructured meshes with complex geometries remains a significant challenge. Existing efficient architectures often employ feature dimensionality reduction strategies, which inadvertently induces Geometric Aliasing, resulting in the loss of critical physical boundary information. To address this, we propose the Physics-Geometry Operator Transformer (PGOT), designed to reconstruct physical feature learning through explicit geometry awareness. Specifically, we propose Spectrum-Preserving Geometric Attention (SpecGeo-Attention). Utilizing a ``physics slicing-geometry injection" mechanism, this module incorporates multi-scale geometric encodings to explicitly preserve multi-scale geometric features while maintaining linear computational complexity $O(N)$. Furthermore, PGOT dynamically routes computations to low-order linear paths for smooth regions and high-order non-linear paths for shock waves and discontinuities based on spatial coordinates, enabling spatially adaptive and high-precision physical field modeling. PGOT achieves consistent state-of-the-art performance across four standard benchmarks and excels in large-scale industrial tasks including airfoil and car designs.

Figures (17)

• Figure 1: Efficiency and accuracy comparison on standard benchmarks. (a) Inference speed vs. Memory usage. Bubble size indicates model size. (b) Multi-dimensional performance metrics on PDE and industrial datasets.
• Figure 2: Overall architecture of PGOT. (a) The framework explicitly integrates multi-scale geometry via stacked PhysGeoBlocks to reconstruct velocity and pressure fields on complex 3D meshes. (b) Visualization of TaylorDecomp-FFN. The Linear Expert (blue) captures smooth conservation dynamics, while the Non-linear Expert (red) targets high-order fluctuations.
• Figure 3: Architecture of PhysGeoBlock and SpecGeo-Attention. (a) The PhysGeoBlock integrates explicit geometric coordinates into the SpecGeo-Attention and TaylorDecomp-FFN layers. (b) The "physics slicing-geometry injection" paradigm. A Spectrum Encoder generates geometry-aware weights to aggregate $N$ mesh points into $M$ latent tokens (Slice) and reconstruct them (DeSlice). This design achieves linear complexity $\mathcal{O}(N)$ while preserving multi-scale geometric fidelity.
• Figure 4: Visualization of prediction errors on standard benchmarks. The first column shows ground truth fields, and the remaining columns display prediction errors.
• Figure 5: Visualization on industrial benchmarks. (a) AirfRANS: ground truth pressure field and prediction errors. (b) Shape-Net Car: ground truth streamlines and PGOT prediction, along with surrounding velocity and surface pressure errors.