Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PDEformer: Towards a Foundation Model for One-Dimensional Partial Differential Equations

Zhanhong Ye, Xiang Huang, Leheng Chen, Hongsheng Liu, Zidong Wang, Bin Dong

TL;DR

PDEformer addresses the challenge of building a foundation model that can solve a broad class of one-dimensional PDEs by explicitly encoding the PDE form as a computational graph and coupling a graph Transformer with an implicit neural representation. The method yields mesh-free predictions conditioned on a learned latent code, enabling zero-shot generalization across PDE types and rapid few-shot fine-tuning, as well as solving inverse coefficient recovery from observed data. Key results show strong pretraining performance with low relative $L^2$ error, competitive forward predictions on PDEBench without task-specific training, and robust coefficient recovery under noise. This work advances the goal of a general-purpose PDE solver with potential applicability to downstream tasks like inverse design and control, marking a milestone toward a foundation PDE model for 1D problems.

Abstract

This paper introduces PDEformer, a neural solver for partial differential equations (PDEs) capable of simultaneously addressing various types of PDEs. We propose to represent the PDE in the form of a computational graph, facilitating the seamless integration of both symbolic and numerical information inherent in a PDE. A graph Transformer and an implicit neural representation (INR) are employed to generate mesh-free predicted solutions. Following pretraining on data exhibiting a certain level of diversity, our model achieves zero-shot accuracies on benchmark datasets that is comparable to those of specifically trained expert models. Additionally, PDEformer demonstrates promising results in the inverse problem of PDE coefficient recovery.

PDEformer: Towards a Foundation Model for One-Dimensional Partial Differential Equations

TL;DR

PDEformer addresses the challenge of building a foundation model that can solve a broad class of one-dimensional PDEs by explicitly encoding the PDE form as a computational graph and coupling a graph Transformer with an implicit neural representation. The method yields mesh-free predictions conditioned on a learned latent code, enabling zero-shot generalization across PDE types and rapid few-shot fine-tuning, as well as solving inverse coefficient recovery from observed data. Key results show strong pretraining performance with low relative error, competitive forward predictions on PDEBench without task-specific training, and robust coefficient recovery under noise. This work advances the goal of a general-purpose PDE solver with potential applicability to downstream tasks like inverse design and control, marking a milestone toward a foundation PDE model for 1D problems.

Abstract

This paper introduces PDEformer, a neural solver for partial differential equations (PDEs) capable of simultaneously addressing various types of PDEs. We propose to represent the PDE in the form of a computational graph, facilitating the seamless integration of both symbolic and numerical information inherent in a PDE. A graph Transformer and an implicit neural representation (INR) are employed to generate mesh-free predicted solutions. Following pretraining on data exhibiting a certain level of diversity, our model achieves zero-shot accuracies on benchmark datasets that is comparable to those of specifically trained expert models. Additionally, PDEformer demonstrates promising results in the inverse problem of PDE coefficient recovery.
Paper Structure (24 sections, 10 equations, 10 figures, 5 tables)

This paper contains 24 sections, 10 equations, 10 figures, 5 tables.

Table of Contents

  1. Introduction and Related Work
  2. Methodology
  3. Graph Construction
  4. Encoding Graph Data
  5. Decoding the PDE Solution
  6. Results
  7. Pretraining stage
  8. Forward problem
  9. Inverse problem
  10. Detailed Intepretation of the Computational Graph Representation
  11. Datasets
  12. PreTraining Datasets
  13. PDEBench Datasets
  14. Network Architecture and Training Setting
  15. Graph Transformer Architecture
  16. ...and 9 more sections

Figures (10)

  • Figure 1: PDEformer architecture, taking $\mathcal{F}(u,c)=u_t+cu_x$ as the example.
  • Figure 2: Comparison of prediction results obtained from the pretrained PDEformer on the test dataset with reference solutions. Each row in the figure represents a single sample, and these three samples were randomly selected.
  • Figure 3: Comparing the speed of fine-tuning PDEformer with training FNO from scratch and fine-tuning a pretrained FNO. The right subfigure uses a logarithmic scale for the $x$-axis, whereas the left employs a linear scale. The vertical lines correspond to $100$ iterations.
  • Figure 4: Results of the PDE coefficient recovery problem under various noise levels. For every PDE, all non-zero coefficients are recovered, with each coefficient depicted as a point in the figure. Consequently, the number of points displayed exceeds the number of PDEs involved.
  • Figure 5: Illustration of how the form of a PDE can be represented as a computational graph, taking the advection equation $u_t+cu_x=0,u(0,x)=g(x)$ as the example. The left panel shows the logical meaning of the nodes and edges, and the right panel illustrates the formalized data structure that is taken as the input of PDEformer. We also note that, different from textual representations, this formalization of DAG is independent of the choice of symbols and the order of addition or multiplication. For example, the equation $\beta v_x+v_t=0,v|_{t=0}=v_0$ also corresponds to the DAG shown on the right panel.
  • ...and 5 more figures