Leveraging Influence Functions for Resampling Data in Physics-Informed Neural Networks

TL;DR

This work addresses training data efficiency in physics-informed neural networks (PINNs) by leveraging training data attribution via influence functions. It introduces PINNfluence, an influence-function-based resampling strategy that scores candidate collocation points and adaptively refines the training set, either by adding or replacing points, using a distribution that emphasizes high-impact samples. Across five PDEs (diffusion, Burgers', Allen-Cahn, wave, drift-diffusion), PINNfluence achieves performance on par with or better than residual-based adaptive refinement (RAR) and other baselines, demonstrating the potential of XAI-inspired data selection to improve PINN training. While offering competitive accuracy, the method incurs higher computational costs due to Hessian and backward-pass calculations, motivating future work on efficiency and integration with broader theory such as neural tangent kernels.

Abstract

Physics-informed neural networks (PINNs) offer a powerful approach to solving partial differential equations (PDEs), which are ubiquitous in the quantitative sciences. Applied to both forward and inverse problems across various scientific domains, PINNs have recently emerged as a valuable tool in the field of scientific machine learning. A key aspect of their training is that the data -- spatio-temporal points sampled from the PDE's input domain -- are readily available. Influence functions, a tool from the field of explainable AI (XAI), approximate the effect of individual training points on the model, enhancing interpretability. In the present work, we explore the application of influence function-based sampling approaches for the training data. Our results indicate that such targeted resampling based on data attribution methods has the potential to enhance prediction accuracy in physics-informed neural networks, demonstrating a practical application of an XAI method in PINN training.

Figures (15)

• Figure 1: Performance comparison between PINN-fluence (solid lines) and RAR (dashed lines) resampling strategies across five PDEs (different colors), with results normalized to uniform random sampling. Lines represent the ratio of $L^2$ relative error (method to random sampling) averaged over 10 runs, with shaded regions indicating standard deviation. Lower values indicate better performance relative to random sampling.
• Figure 2: Comparison of different scoring functions for resampling on five PDEs for iteratively adding new data points. Lines represent $L^2$ relative error averaged over 10 runs, with each point representing the model state after one cycle of Adam+LBFGS, and shaded regions indicating log-scale standard deviations.
• Figure 3: Comparison of different scoring functions for resampling on five PDEs for iteratively replacing the whole training set. Lines represent $L^2$ relative error averaged over 10 runs, with each point representing the model state after one cycle of Adam+LBFGS, and shaded regions indicating log-scale standard deviations.
• Figure A.1: Diffusion (Adding): Training point selection dynamics throughout training across respective scoring methods, averaged over 10 runs. Black points represent sampling probability determined by \ref{['eq:res-pmf']} with $\alpha=2$ and $c=0$. High transparency indicates low sampling probability. Values are clipped to $[-1,1]$.
• Figure A.2: Diffusion (Replacing): Training point selection dynamics throughout training across respective scoring methods, averaged over 10 runs. Black points represent sampling probability determined by \ref{['eq:res-pmf']} with $\alpha=1$ and $c=1$. High transparency indicates low sampling probability. Values are clipped to $[-1,1]$.