DeltaPhi: Physical States Residual Learning for Neural Operators in Data-Limited PDE Solving

TL;DR

DeltaPhi tackles data scarcity in neural operators for PDE solving by reframing learning as residuals between similar physical states. It introduces a residual operator $\mathcal{G}^{\Delta}$ and an auxiliary-sample retriever $\mathcal{F}^{\text{ret}}$, enabling implicit data augmentation and improved generalization across regular/irregular domains and cross-resolution settings. The method is architecture-agnostic and demonstrates consistent gains across multiple base operators (e.g., FNO, FFNO, CFNO) and problems (e.g., Darcy Flow, Navier–Stokes), with notable improvements in data-limited regimes and zero-shot high-resolution generalization. This residual learning framework offers practical benefits for scientific ML and suggests broader applicability to other high-dimensional regression tasks governed by stable physical dynamics.

Abstract

The limited availability of high-quality training data poses a major obstacle in data-driven PDE solving, where expensive data collection and resolution constraints severely impact the ability of neural operator networks to learn and generalize the underlying physical system. To address this challenge, we propose DeltaPhi, a novel learning framework that transforms the PDE solving task from learning direct input-output mappings to learning the residuals between similar physical states, a fundamentally different approach to neural operator learning. This reformulation provides implicit data augmentation by exploiting the inherent stability of physical systems where closer initial states lead to closer evolution trajectories. DeltaPhi is architecture-agnostic and can be seamlessly integrated with existing neural operators to enhance their performance. Extensive experiments demonstrate consistent and significant improvements across diverse physical systems including regular and irregular domains, different neural architectures, multiple training data amount, and cross-resolution scenarios, confirming its effectiveness as a general enhancement for neural operators in data-limited PDE solving.

Figures (5)

• Figure 1: The overall architecture of Physical States Residual Learning. Given an input function$a_i$, we first sample a similar auxiliary sample$(a_{k_i},u_{k_i})$ from the training sample set$\mathcal{T}$. Subsequently, $a_i$ and $(a_{k_i},u_{k_i})$ are concatenated and fed into the the neural operator $\mathcal{G}_{\theta}^{\Delta}$, producing the predicted states residual. Finally, the predicted solution $\hat{u}_i$ is obtained by adding the predicted states residual with the auxiliary solution $u_{k_i}$.
• Figure 2: Prediction error visualization on irregular domain problems.
• Figure 3: Training curve comparison. "DP-*" denotes the proposed residual learning version of base models.
• Figure 4: The curve of distance between $u^{test}(x)$ and $u_{k_t}(x)$ as retrieval similarity rank increases.
• Figure 5: Label distribution visualization. The points represent dimension-reduced labels ($u_*(x)$ for direct learning, $u_*(x)-u_{k_*}(x)$ for residual learning) through Principal Component Analysis. The ellipses with dotted lines and solid lines represent standard deviation and range, respectively. Green and red color correspond to training and testing set, respectively.