A Physics-preserved Transfer Learning Method for Differential Equations

TL;DR

A Physics-preserved Optimal Tensor Transport (POTT) method that simultaneously admits generalizability to common DEs and physics preservation of specific problem is proposed to adapt the data-driven model to target domain utilizing the push-forward distribution induced by the POTT map.

Abstract

While data-driven methods such as neural operator have achieved great success in solving differential equations (DEs), they suffer from domain shift problems caused by different learning environments (with data bias or equation changes), which can be alleviated by transfer learning (TL). However, existing TL methods adopted in DEs problems lack either generalizability in general DEs problems or physics preservation during training. In this work, we focus on a general transfer learning method that adaptively correct the domain shift and preserve physical information. Mathematically, we characterize the data domain as product distribution and the essential problems as distribution bias and operator bias. A Physics-preserved Optimal Tensor Transport (POTT) method that simultaneously admits generalizability to common DEs and physics preservation of specific problem is proposed to adapt the data-driven model to target domain utilizing the push-forward distribution induced by the POTT map. Extensive experiments demonstrate the superior performance, generalizability and physics preservation of the proposed POTT method.

Figures (5)

• Figure 1: Visualization of DA method (COD) in task $\mathcal{D}_3 \to \mathcal{D}_2$ on Darcy flow. The 1st and 2nd columns present the input $k(x)$ and output $u(x)$ sample pairs of $\mathcal{D}_3$ and $\mathcal{D}_2$. The 3rd column visualizes their feature maps from the aligned distributions, which are forced to be analogous but lacks clear structure explicit physical meaning. It is unclear whether they retain the correct physical information. The prediction shown in the 4th column verifies that the physical structures of $u$ are not fully preserved.
• Figure 2: Illustration of POTT. Left: Illustration of problem formulation. The distribution bias leads to the bias of physics, i.e. the operator bias. The goal is to correct the operator bias. Right: POTT correct the operator bias by characterizing ${\mathcal{D}^t}$ in a physics-preserved way. (0) Before the model transfer process, source model $\hat{\mathcal{G}^s}$ is pretrained with sufficient source data. (1) The POTT map $T_\theta$ between source and target domain is learned. (2) The target distribution $P^t$ is characterized by the pushforward distribution $P^r$. (3) $\hat{\mathcal{G}^s}$ is transferred to $\hat{\mathcal{G}^t}$ with $\hat{\mathcal{D}^t}$ and $\hat{\mathcal{D}^r}$.
• Figure 3: Comparision with (1) Source-only, source pretrained model, (2) Finetuning on target domain, (3) ClimaX, a SOTA climate forecasting model, trained with sufficient target data, (4) ClimODE, the backbone model, trained with sufficient target data. The light area reflects the standard deviation of RMSE $(\downarrow )$. Details are provided in Tab. \ref{['tab:climate']}.
• Figure 4: Visualization of $\hat{\mathcal{G}}_t$ on Darcy Flow. The value at each image pixel represents the function value at the sampling point. Brighter colors (yellow) indicate higher values. Columns 1-2 show the input-output function pairs from the target domain. Columns 3-5 show the output of POTT, finetuning (F.T.) and COD. Columns 6-8 display the prediction errors relative to the ground truth $u(x)$.
• Figure 5: Visualization of POTT and OTT.

Theorems & Definitions (4)

• Definition 4.1: POTT
• Theorem 4.2: Consistency
• Theorem B.1: Consistency
• proof : Proof