Adaptive Physics-informed Neural Networks: A Survey

TL;DR

Adaptive PINNs aim to overcome data scarcity and costly retraining by leveraging transfer learning and meta-learning to efficiently adapt to families of related PDEs. The survey organizes methods into full fine-tuning, parameter-efficient fine-tuning, curriculum transfer learning, and various meta-learning strategies that learn weight initializations, network structures, losses, and inputs, as well as few-shot approaches. It surveys benchmark PDEs (notably Burgers and Poisson), error metrics (including worst-case task errors), and efficiency metrics (data and computation budgets), and discusses applications, limitations, and alignment with conventional solvers. The work highlights promising directions such as GPT-PINN, SVD-PINN, hypernetwork-based adaptivity, and adaptive sampling, while underscoring the need for standardized evaluation frameworks to enable fair comparisons and broader adoption in science and engineering.

Abstract

Physics-informed neural networks (PINNs) have emerged as a promising approach to solving partial differential equations (PDEs) using neural networks, particularly in data-scarce scenarios, due to their unsupervised training capability. However, limitations related to convergence and the need for re-optimization with each change in PDE parameters hinder their widespread adoption across scientific and engineering applications. This survey reviews existing research that addresses these limitations through transfer learning and meta-learning. The covered methods improve the training efficiency, allowing faster adaptation to new PDEs with fewer data and computational resources. While traditional numerical methods solve systems of differential equations directly, neural networks learn solutions implicitly by adjusting their parameters. One notable advantage of neural networks is their ability to abstract away from specific problem domains, allowing them to retain, discard, or adapt learned representations to efficiently address similar problems. By exploring the application of these techniques to PINNs, this survey identifies promising directions for future research to facilitate the broader adoption of PINNs in a wide range of scientific and engineering applications.

Figures (4)

• Figure 1: Data and scientific knowledge requirements for different modeling approaches.
• Figure 2: Schematic representation of PINNs: The network is optimized by minimizing a composite loss function that combines a regression loss from observed data, PDE residuals at collocation points, and boundary/initial condition losses.
• Figure 3: Illustration of IBVP as tasks: a) Heat equation tasks with varying material properties. b) Burgers' equation tasks with different initial conditions (adapted from takamoto2022pdebench).
• Figure 4: Example of efficient model adaptation through meta-learning. The blue-circled metrics represent key factors that efficient adaptivity aims to reduce, influenced by the other metrics in the radar chart. On the far right is the predicted solution after only 100 epochs.