PINNACLE: PINN Adaptive ColLocation and Experimental points selection

TL;DR

This work tackles the challenge of training physics-informed neural networks (PINNs) under practical point-budget constraints by jointly optimizing the selection of experimental points and collocation points for PDEs and IC/BCs. It introduces PINNACLE, an NTK-informed framework that operates in an augmented input space ${\mathcal{Z}}$ to capture cross-information among point types, and defines a convergence degree $\alpha(Z)$ to guide point selection. A Nyström-based approximation computes the eNTK spectrum efficiently, enabling two PINNACLE variants (PINNACLE-S and PINNACLE-K) that alternate between point selection and training. Theoretical connections show that maximizing $\alpha(Z)$ reduces generalization error, and empirical results across forward, inverse, and transfer learning tasks demonstrate that PINNACLE consistently outperforms baselines and yields interpretable, adaptive point distributions. The approach promises improved efficiency for PINNs and could be extended to broader deep learning problems with composite losses and multi-domain inputs.

Abstract

Physics-Informed Neural Networks (PINNs), which incorporate PDEs as soft constraints, train with a composite loss function that contains multiple training point types: different types of collocation points chosen during training to enforce each PDE and initial/boundary conditions, and experimental points which are usually costly to obtain via experiments or simulations. Training PINNs using this loss function is challenging as it typically requires selecting large numbers of points of different types, each with different training dynamics. Unlike past works that focused on the selection of either collocation or experimental points, this work introduces PINN Adaptive ColLocation and Experimental points selection (PINNACLE), the first algorithm that jointly optimizes the selection of all training point types, while automatically adjusting the proportion of collocation point types as training progresses. PINNACLE uses information on the interaction among training point types, which had not been considered before, based on an analysis of PINN training dynamics via the Neural Tangent Kernel (NTK). We theoretically show that the criterion used by PINNACLE is related to the PINN generalization error, and empirically demonstrate that PINNACLE is able to outperform existing point selection methods for forward, inverse, and transfer learning problems.

Figures (24)

• Figure 1: Reconstruction of the PINN prediction values $F[\hat{u}_{\theta}](z)$, after 10k GD training steps, in $(x, \mathrm{s})$ and $(x,\mathrm{p})$ subspaces of the 1D Advection equation, using the dominant eNTK eigenfunctions (i.e., those with the largest eigenvalues).
• Figure 2: Reconstruction of the true PDE solution of the 1D Advection equation using eigenfunctions of $\Theta_t$ at different GD timesteps. For each time step, we plot the eigenfunction component both for $(x, \mathrm{s})$ subspace (top row) and $(x, \mathrm{p})$ subspace (bottom row).
• Figure 3: Example of points selected by PINNACLE-K during training of various forward problems. The points represent the various types of CL points selected by PINNACLE-K, whereas the patterns in the background represents the obtained NN solution.
• Figure 4: Results from the forward problem experiments. In each row, from left to right: plot of mean prediction error for each method, the true PDE solution, and the PINN output from different point selection methods.
• Figure 5: The two inverse parameters learned in the 2D Navier-Stokes equation problem. The black horziontal lines represents the true parameter values, while the other lines represent the predicted values of the inverse parameters during training.

Theorems & Definitions (22)

• Theorem 1: Generalization Bound, Informal Version of \ref{['thm:generalisation_pinn']}
• Proposition 1: Criterion Approximation Bound, Informal Version of \ref{['corr:nystrom-good']}
• Lemma 1
• proof
• Corollary 1
• proof
• Lemma 2
• proof
• Lemma 3
• proof