From Cheap Geometry to Expensive Physics: Elevating Neural Operators via Latent Shape Pretraining

TL;DR

The paper tackles the data-scarcity challenge in neural operators for PDEs by introducing a two-stage training framework that exploits abundant geometry-only data. In Stage 1, a physics-agnostic pretraining step trains a point-cloud VAE to learn latent geometry representations via an occupancy-field proxy; in Stage 2, neural operators are trained to map these learned latents to PDE solutions, freezing the encoder. Across four PDE datasets and three transformer-based operators, the approach yields consistent accuracy gains, especially for query batches drawn from random locations, demonstrating that geometry-informed latent representations enhance data efficiency. The method offers a flexible, scalable path to improve surrogate PDE solvers in industrial settings where labeled physics data are scarce but geometry samples are plentiful.

Abstract

Industrial design evaluation often relies on high-fidelity simulations of governing partial differential equations (PDEs). While accurate, these simulations are computationally expensive, making dense exploration of design spaces impractical. Operator learning has emerged as a promising approach to accelerate PDE solution prediction; however, its effectiveness is often limited by the scarcity of labeled physics-based data. At the same time, large numbers of geometry-only candidate designs are readily available but remain largely untapped. We propose a two-stage framework to better exploit this abundant, physics-agnostic resource and improve supervised operator learning under limited labeled data. In Stage 1, we pretrain an autoencoder on a geometry reconstruction task to learn an expressive latent representation without PDE labels. In Stage 2, the neural operator is trained in a standard supervised manner to predict PDE solutions, using the pretrained latent embeddings as inputs instead of raw point clouds. Transformer-based architectures are adopted for both the autoencoder and the neural operator to handle point cloud data and integrate both stages seamlessly. Across four PDE datasets and three state-of-the-art transformer-based neural operators, our approach consistently improves prediction accuracy compared to models trained directly on raw point cloud inputs. These results demonstrate that representations from physics-agnostic pretraining provide a powerful foundation for data-efficient operator learning.

Figures (7)

• Figure 1: Illustration of the proposed two-stage training framework. Stage 1 leverages the abundant geometry data for pretraining. Stage 2 learns PDE solutions on the scarce physics data.
• Figure 2: An illustration of our two-stage training framework. In stage 1, an encoder is pretrained on rich geometry data using a physics-agnostic proxy task, for which we choose occupancy field. The encoded latent representation is then integrated to transformer-based neural operators for stage 2 training on scarce PDE solution labels. The performance is tested on two query sampling strategies: mesh-based query, and uniform random queries.
• Figure 3: Visualization of a sample data point from the near volume AirfRans dataset. All of the datasets are preprocessed so that each data point consists of four types of information: physics field queried on the mesh point cloud, physics field queried on randomly sampled points, occupancy field queried on randomly sampled points, and occupancy field queried on the mesh point cloud perturbed by random displacements. The geometry data $D'$ are labeled only with occupancy fields.
• Figure 4: Examples of physics data and geometry data from the Stress dataset. Neural Operators only learn from von Mises stress fields on type 1 void, while observing the geometry data from geometries generated from 4 types of different void priors.
• Figure 5: Example of a 3D inductor data. We generate the geometries (iron core in purple, coil in yellow) by parameterizing an EE type iron core, and calculate the corresponding magnetic flux density norm field (on the right).