Pretrain Finite Element Method: A Pretraining and Warm-start Framework for PDEs via Physics-Informed Neural Operators

TL;DR

The paper introduces PFEM, a two-stage framework that pretrains a Transolver-based physics-informed neural operator on unstructured point clouds using PDE constraints alone, yielding physically consistent, fast initial field predictions. In the warm-start stage, these predictions initialize conventional FEM solvers, dramatically reducing iterations while preserving FEM accuracy, applicable to linear and nonlinear elasticity with complex geometries and heterogeneous materials. PFEM demonstrates strong discretization-invariant generalization and data-efficient training, supported by patch-test diagnostics, out-of-distribution robustness, and self-improvement potential as more scenarios are encountered. The approach offers a cloud-based pretraining plus local refinement workflow, enabling significant computational savings without sacrificing numerical fidelity, and points to future extensions to transient problems and mesh-free implementations.

Abstract

We propose a Pretrained Finite Element Method (PFEM),a physics driven framework that bridges the efficiency of neural operator learning with the accuracy and robustness of classical finite element methods (FEM). PFEM consists of a physics informed pretraining stage and an optional finetuning stage. In the pretraining stage, a neural operator based on the Transolver architecture is trained solely from governing partial differential equations, without relying on labeled solution data. The model operates directly on unstructured point clouds, jointly encoding geometric information, material properties, and boundary conditions, and produces physically consistent initial solutions with extremely high computational efficiency. PDE constraints are enforced through explicit finite element, based differentiation, avoiding the overhead associated with automatic differentiation. In the fine-tuning stage, the pretrained prediction is used as an initial guess for conventional FEM solvers, preserving their accuracy, convergence guarantees, and extrapolation capability while substantially reducing the number of iterations required to reach a prescribed tolerance. PFEM is validated on a broad range of benchmark problems, including linear elasticity and nonlinear hyperelasticity with complex geometries, heterogeneous materials, and arbitrary boundary conditions. Numerical results demonstrate strong generalization in the pretraining stage with relative errors on the order of 1\%, and speedups of up to one order of magnitude in the fine-tuning stage compared to FEM with zero initial guesses.

Figures (20)

• Figure 1: Illustration of the input and output representation in the Fourier Neural Operator (FNO).
• Figure 2: The architecture of Transolver layer
• Figure 3: Schematic illustration of the Pretrained Finite Element Method (PFEM). PFEM consists of a pretraining stage and a warm-start stage. In the pretraining stage, a Transolver-based physics-informed neural operator (PINO) is trained using governing physical equations; the dashed line indicates that data-driven training is optional but not required. This stage yields a low-fidelity solution with extremely high efficiency. Spatial derivatives are constructed via explicit function differentiation, and either strong-form or variational PDE losses are employed. In the warm-start stage, classical numerical solvers (e.g., FEM, finite difference, or finite volume methods) are cast into iterative forms, and the neural operator prediction is used as the initial guess to obtain a high-fidelity solution.
• Figure 4: Schematic illustration of the warmstart phase of PFEM.
• Figure 5: Elastic plate with arbitrary holes: (a) randomly generated hole geometries in the dataset; (b) boundary condition configuration; (c) comparison of input representations, where FNO requires structured grids while Transolver operates directly on point clouds.