PDEformer-1: A Foundation Model for One-Dimensional Partial Differential Equations

TL;DR

PDEformer-1 introduces a foundation-model approach to solving one-dimensional PDEs by representing the PDE as a computational graph and using a graph Transformer to encode symbolic structure alongside numeric data. An INR decoder then produces mesh-free solutions, enabling zero-shot inference and rapid finetuning, as well as solving inverse problems like coefficient and source-field recovery. The model is pretrained on a large, diverse 1D PDE dataset and demonstrates strong in-distribution performance, competitive inference speed, and notable adaptability to unseen PDEs and OoD scenarios with limited fine-tuning data. The work highlights the potential of a unified, scalable PDE solver that leverages symbolic-numeric fusion and mesh-free decoding, with future plans to extend to higher dimensions and more complex PDE forms.

Abstract

This paper introduces PDEformer-1, a versatile neural solver capable of simultaneously addressing various partial differential equations (PDEs). With the PDE represented as a computational graph, we facilitate the seamless integration of symbolic and numeric information inherent in a PDE. A graph Transformer and an implicit neural representation (INR) are employed subsequently to generate mesh-free predicted solutions. We generated a dataset with up to three million samples involving diverse one-dimensional PDEs to pretrain our model. Compared with baseline models trained specifically on benchmark datasets, our pretrained model achieves comparable accuracy via zero-shot inference, and the advantage expands after finetuning. For PDEs new or unseen in the pretraining stage, our model can adapt quickly by finetuning on a relatively small set of examples from the target equation. Additionally, PDEformer-1 demonstrates promising results in the inverse problem of PDE scalar coefficient recovery and coefficient field recovery.

Figures (21)

• Figure 1: Overall architecture of PDEformer-1, taking the Advection equation $u_t+cu_x=0$, $u(0,x)=g(x)$ with periodic boundary condition as the example. Since only one unknown field variable $u_0=u$ is involved in this equation, we omit the subscript $j=0$ for $u_j$ and $\pmb{\mu}_j$ in the figure.
• Figure 2: An illustrative comparison of the grids for periodic and non-periodic PDEs in our dataset, in which a resolution of $16$ points is used for simplicity. Thanks to the mesh-free nature of PDEformer-1, we can utilize solution samples recorded on different grid points directly during training, and no interpolation is required.
• Figure 3: The figure on the left shows the change of test accuracy according to compute time. The figure on the right shows the change of test accuracy according to training dataset size. Different lines refers to different scales of model or training dataset. For example, M-30k represents training the M-size model on a dataset with 30k data samples.
• Figure 4: Inference performance of PDEformer-1 on part of the in-distribution test set. The horizontal axis represents the temporal axis $t$, and the vertical axis represents the spatial axis $x$.
• Figure 5: Variation of test error with number of finetuning samples. 'PDEformer-1 (Zero-Shot)' represents our model's direct inference capability without any finetuning. This unique characteristic is visually depicted as a horizontal dashed line across the figures.