How to Re-enable PDE Loss for Physical Systems Modeling Under Partial Observation

TL;DR

RPLPO introduces a principled framework to re-enable PDE loss for physical system modeling under partial observation by jointly learning a learnable high-resolution state and its forward transition. The encoding module reconstructs high-resolution states from partial observations, while the transition module predicts future high-resolution states, with PDE constraints applied to adjacent high-resolution pairs. A two-period training strategy—base training on labeled data and two-stage fine-tuning with unlabeled data—enables robust generalization even when data are scarce, noisy, or when PDEs are imperfect. Across five PDE benchmarks, RPLPO outperforms strong baselines in single- and multi-step predictions, demonstrating improved reconstruction fidelity, physical consistency, and computational practicality for sensor-limited scenarios.

Abstract

In science and engineering, machine learning techniques are increasingly successful in physical systems modeling (predicting future states of physical systems). Effectively integrating PDE loss as a constraint of system transition can improve the model's prediction by overcoming generalization issues due to data scarcity, especially when data acquisition is costly. However, in many real-world scenarios, due to sensor limitations, the data we can obtain is often only partial observation, making the calculation of PDE loss seem to be infeasible, as the PDE loss heavily relies on high-resolution states. We carefully study this problem and propose a novel framework named Re-enable PDE Loss under Partial Observation (RPLPO). The key idea is that although enabling PDE loss to constrain system transition solely is infeasible, we can re-enable PDE loss by reconstructing the learnable high-resolution state and constraining system transition simultaneously. Specifically, RPLPO combines an encoding module for reconstructing learnable high-resolution states with a transition module for predicting future states. The two modules are jointly trained by data and PDE loss. We conduct experiments in various physical systems to demonstrate that RPLPO has significant improvement in generalization, even when observation is sparse, irregular, noisy, and PDE is inaccurate.

Figures (9)

• Figure 1: RPLPO. Training (left): In the base-training period, the encoding module is trained by $\mathcal{L}_D$ and $\mathcal{L}_P^\theta$, and the transition module is trained by $\mathcal{L}_D$ and $\mathcal{L}_P^\phi$. These losses are calculated on labeled dataset $\mathcal{D}$. Then, in the two-stage fine-tuning period, the transition module is tuned by $\mathcal{L}_P^\phi$, calculated on unlabeled dataset $\mathcal{B}$, and the encoding module is tuned by $\mathcal{L}_D$, calculated on $\mathcal{D}$, in order. Inference (right): given the partial observation to predict future partially observed states. For the two-stage fine-tuning period, please check Technical Appendix Figure 4.
• Figure 2: The performance of multi-steps prediction from 1st to 10th step. Our proposed RPLPO achieves a significant improvement against FNO* on five benchmarks among all prediction steps.
• Figure 3: Relative loss $\mathcal{L}_{D}$($\downarrow$) of three types hyperparameters. In the right one, we omit "$m_2$" on x-coordinate means $m_2=m_1$.
• Figure 4: Two-Stage Fine-Tuning Period. The first stage (left) and the second stage (right).
• Figure 5: An example of irregular partially observed positions for nine number of points. The orange points are the irregular observations and the black region is unobservable.