PI-MFM: Physics-informed multimodal foundation model for solving partial differential equations

TL;DR

The paper addresses data-efficient, transferable PDE solving across diverse equation families by introducing PI-MFM, a physics-informed multimodal foundation model that encodes PDEs symbolically and automatically assembles PDE residuals for training. It demonstrates that enforcing physics Losses alongside limited labeled data yields superior performance, robustness to noise, and strong generalization across 13 1D time-dependent PDE families, including zero-shot adaptation to unseen PDEs. The work provides a practical framework for vectorized derivative computation, compares automatic differentiation and finite differences under various precisions, and shows that zero-shot physics-informed fine-tuning can reach about 1% L2 relative error on new PDE families. These findings highlight PI-MFM as a scalable, data-efficient approach to transfer-ready PDE solvers, with guidelines for differentiation backends, collocation sampling, and future extensions to higher dimensions and uncertainty quantification.

Abstract

Partial differential equations (PDEs) govern a wide range of physical systems, and recent multimodal foundation models have shown promise for learning PDE solution operators across diverse equation families. However, existing multi-operator learning approaches are data-hungry and neglect physics during training. Here, we propose a physics-informed multimodal foundation model (PI-MFM) framework that directly enforces governing equations during pretraining and adaptation. PI-MFM takes symbolic representations of PDEs as the input, and automatically assembles PDE residual losses from the input expression via a vectorized derivative computation. These designs enable any PDE-encoding multimodal foundation model to be trained or adapted with unified physics-informed objectives across equation families. On a benchmark of 13 parametric one-dimensional time-dependent PDE families, PI-MFM consistently outperforms purely data-driven counterparts, especially with sparse labeled spatiotemporal points, partially observed time domains, or few labeled function pairs. Physics losses further improve robustness against noise, and simple strategies such as resampling collocation points substantially improve accuracy. We also analyze the accuracy, precision, and computational cost of automatic differentiation and finite differences for derivative computation within PI-MFM. Finally, we demonstrate zero-shot physics-informed fine-tuning to unseen PDE families: starting from a physics-informed pretrained model, adapting using only PDE residuals and initial/boundary conditions, without any labeled solution data, rapidly reduces test errors to around 1% and clearly outperforms physics-only training from scratch. These results show that PI-MFM provides a practical and scalable path toward data-efficient, transferable PDE solvers.

Figures (8)

• Figure 1: Overview of the physics-informed multimodal foundation model (PI-MFM) framework for multi-operator learning (MOL). (A) A representative example of PDE-encoding multi-operator learning framework based on PROSE liu2024prosesun2025towards. It jointly encodes data inputs and symbolic PDE expressions, fuses these modalities, and decodes solution values at arbitrary query points $(t,x)$. (B) PI-MFM training workflow. Given a symbolic PDE, the framework automatically selects and computes the required derivatives of the PDE solution and assembles PDE-residual losses and data losses into a total loss. This workflow can be applied to any MFM that explicitly accepts PDE operator information. For simplicity, IC losses are omitted in the workflow and can be viewed as special cases of data or PDE losses. Periodic boundary conditions are enforced by augmenting the input coordinates with periodic features lu2021physicslu2022comprehensive, so that periodicity is built into the network rather than imposed through an explicit BC loss term.
• Figure 2: Comparison of PI-MFM (w/ physics) and purely data-driven (w/o physics) models in the sparse data regime. (A) An illustrative SG example under increasing resolution. (B) A representative Cons-Sin example from the test set, where the models are trained on labeled data of $8 \times 32$ resolution. The physics-informed model accurately predicts the solution and its derivatives, while the purely data-driven model fails, resulting in large errors. (C) $L^2$ and $H^1$ relative test errors for physics-informed and purely data-driven models. (Left) At each labeled resolution, the mean test error over all 10 PDE families, with shaded bands indicating the corresponding standard deviation across PDEs. (Right) Distribution of errors for each of the 10 PDE families at $4\times16$ resolution.
• Figure 3: Comparison of PI-MFM (w/ physics) and purely data-driven (w/o physics) models in the temporal extrapolation regime. (A) An illustrative Diff-Log example under increasing labeled temporal domain. (B, C, and D) A representative Cons-Cub example from the test set, where the models are trained on labeled data from the temporal domain $[0, 1/2]$. (B) Solution $u$. (C) Temporal derivative $u_t$. (D) Spatial derivative $u_x$. The physics-informed model accurately extrapolates the solution and its derivatives beyond the training domain, while the purely data-driven model fails. (E) Mean $L^2$ and $H^1$ relative test errors over the full temporal domain versus the temporal span of labeled training data. The physics-informed model achieves significantly lower error than the purely data-driven model. (F) Mean $L^2$ relative error over the full temporal domain for models trained on temporal spans of 1/8, 1/4, and 1/2, respectively, highlighting the low and stable errors in both the interpolation (In.) and extrapolation (Ex.) regimes for physics-informed models.
• Figure 4: Comparison of PI-MFM (w/ physics) and purely data-driven (w/o physics) models with varying numbers of labeled function pairs. (A) Representative test-set examples across 10 PDE families, with models trained on 100 labeled function pairs per family. The physics-informed model provides more accurate predictions across the PDE families. (B) Mean $L^2$ and $H^1$ relative test errors on the test dataset versus the number of labeled input-output function pairs per PDE family $N_\text{func}$. The physics-informed model ('w/ Phys.') significantly outperforms the data-driven model ('w/o Phys.') when $N_\text{func}$ is small. (C) Mean $L^2$ relative error on the validation dataset versus training iterations for $N_\text{func}$ of 10, 50, and 100, respectively, showing that the physics-informed model's error continues to decrease while the data-driven model's error plateaus, particularly with fewer function pairs.
• Figure 5: Robustness to data noise. (A) An illustrative Cons-Cub example with increasing label noise level $\gamma$ from 0 to 5. (B) The first figure reports mean $L^{2}$ relative test error versus $\gamma$ for $N_{\text{func}}=1000$ and $50000$. The remaining four figures show training curves at $\gamma\in\{0.5,1,2,5\}$, where larger $N_{\text{func}}$ avoids overfitting. (C) Fixing $\gamma=1$. The first figure shows mean $L^{2}$ relative test error versus $N_{\text{func}}$. The remaining four figures compare data-driven ('w/o Phys.') and physics-informed ('w/ Phys.') training curves at $N_{\text{func}}\in\{100,500,1000,2000\}$, highlighting the stability of physics-informed training.