One-Shot Transfer Learning of Physics-Informed Neural Networks

TL;DR

The paper presents a general, model-agnostic framework for one-shot transfer learning with Physics-Informed Neural Networks (PINNs) applied to linear ODEs and PDEs. By freezing the hidden representation and solving for the final-output weights analytically, it achieves instantaneous, high-fidelity solutions for unseen differential equations, significantly reducing training overhead. The approach is validated on several canonical problems, including first- and second-order ODEs, coupled ODE systems, Poisson, and time-dependent Schrödinger equations, while also addressing numerical conditioning via regularization or QR decomposition. These results imply substantial practical gains for rapid, scalable inference across a broad class of linear differential equations and suggest avenues for extending to nonlinear and data-driven regimes.

Abstract

Solving differential equations efficiently and accurately sits at the heart of progress in many areas of scientific research, from classical dynamical systems to quantum mechanics. There is a surge of interest in using Physics-Informed Neural Networks (PINNs) to tackle such problems as they provide numerous benefits over traditional numerical approaches. Despite their potential benefits for solving differential equations, transfer learning has been under explored. In this study, we present a general framework for transfer learning PINNs that results in one-shot inference for linear systems of both ordinary and partial differential equations. This means that highly accurate solutions to many unknown differential equations can be obtained instantaneously without retraining an entire network. We demonstrate the efficacy of the proposed deep learning approach by solving several real-world problems, such as first- and second-order linear ordinary equations, the Poisson equation, and the time-dependent Schrodinger complex-value partial differential equation.

Figures (7)

• Figure 1: Predicted (colored) versus ground truth (dashed black) phase space, namely a plot of space against velocity for different times, to 20 second-order non-homogeneous ordinary differential equations. Average residuals are shown in the bottom panel.
• Figure 2: Phase space trajectories of the coupled oscillator system for fixed mass and spring constants (top) and spatial solutions (middle). One solution that induces beats is highlighted in color while the other solutions appear in grey. The average residuals of the total realizations are shown in the bottom panel. The initial state of the masses influences how close the normal mode frequencies get. Our network can identify solutions to all 100 initial conditions in $\sim 10^{-2}$ seconds.
• Figure 3: Top: phase space of predicted trajectories of a nonlinear oscillator system. The training curves ares shown in green and the test in blue. Dashed black lines represent ground truth solutions. Bottom: average residuals of 30 predicted solutions across different initial conditions.
• Figure 4: Predicted solution (top) of the Poisson equation with an initial charge distribution $\rho(x,y)$ composed of multiple frequencies $k$. The network is pre-trained on the individual frequencies and can obtain the solution to the combination in one-shot (35s) with high fidelity/low MSE (bottom).
• Figure 5: MSE between predicted and analytic solutions $|\psi|^2$ as a function of $\sigma$ and $p_0$. Red circles represent the three configurations for which the network was batch trained. We see that as $p_0$ increases, transfer learning ${W_\text{out} }$ becomes less effective because of the F-principle bottleneck for PINNs.