Self-adaptive weighting and sampling for physics-informed neural networks

TL;DR

PINNs solve PDEs by minimizing a sum of physics and boundary losses, but training is hampered by mismatched loss scales and uneven point distribution. The authors introduce a hybrid framework that combines BRDR-based self-adaptive weighting with residual-based self-adaptive sampling, balancing convergence across points and concentrating samples where the solution is difficult. The BRDR mechanism uses $IRDR = \frac{R^2}{\sqrt{\overline{R^4}+\epsilon}}$ to reweight contributions and maintain a stable mean weight, while adaptive sampling targets high-residual regions. Across four benchmark problems, the combined approach delivers the best accuracy with modest overhead, improving data efficiency and robustness of PINNs for challenging PDEs, and code will be released for reproducibility.

Abstract

Physics-informed deep learning has emerged as a promising framework for solving partial differential equations (PDEs). Nevertheless, training these models on complex problems remains challenging, often leading to limited accuracy and efficiency. In this work, we introduce a hybrid adaptive sampling and weighting method to enhance the performance of physics-informed neural networks (PINNs). The adaptive sampling component identifies training points in regions where the solution exhibits rapid variation, while the adaptive weighting component balances the convergence rate across training points. Numerical experiments show that applying only adaptive sampling or only adaptive weighting is insufficient to consistently achieve accurate predictions, particularly when training points are scarce. Since each method emphasizes different aspects of the solution, their effectiveness is problem dependent. By combining both strategies, the proposed framework consistently improves prediction accuracy and training efficiency, offering a more robust approach for solving PDEs with PINNs.

Figures (11)

• Figure 1: (Left) PINN prediction for the 1D perturbation equation, and (right) residuals at the training points compared with those on a fine grid.
• Figure 2: Schematic illustration of the implementation process for self-adaptive sampling based on residuals.
• Figure 3: PINN prediction errors (left panels) and training time cost (right panels) for the perturbation, Allen–Cahn, Burgers’ equations, and lid-driven flow. Shaded areas denote the mean ± 2 standard deviations, calculated from 5 independent runs for each case.
• Figure 4: Perturbation problem: (top) PINN prediction errors and exact solution and (bottom) weight distribution and residual point distribution.
• Figure 5: Weighted density estimation for Allen-Cahn equation. "n1-n4" denotes the number of training points 2000, 5000, 10000, 25000, respectively.