GAS: A Gaussian Mixture Distribution-Based Adaptive Sampling Method for PINNs

TL;DR

The paper tackles improving PINN accuracy for PDEs with singularities and high dimensions by introducing GAS, a Gaussian mixture distribution-based adaptive sampling method. GAS constructs a Gaussian mixture density $\rho_{add}(\boldsymbol{x};\eta^k)=\sum_{i=1}^{N_G}\pi_i^k\mathcal{N}(\boldsymbol{x}|\mu_i^k,\Sigma_i^k)$ guided by the current residual $r(\boldsymbol{x};\Theta)$, using Laplace approximation to set means $\mu_i^k$ and covariances $\Sigma_i^k$, and updates the sampling density via $\rho_{k+1}=\alpha_k\rho_k+(1-\alpha_k)\rho_{add}$. The method frames adaptive sampling as a min-max problem and employs incremental learning to progressively refine the model while avoiding forgetting. Through experiments on 2D and 10D problems including single, multi-peak, and high-dimensional linear PDEs, GAS achieves state-of-the-art or competitive accuracy with fewer samples compared to uniform sampling and existing adaptive methods, demonstrating its practical impact for efficient PINN training in complex regimes.

Abstract

With the recent study of deep learning in scientific computation, the Physics-Informed Neural Networks (PINNs) method has drawn widespread attention for solving Partial Differential Equations (PDEs). Compared to traditional methods, PINNs can efficiently handle high-dimensional problems, but the accuracy is relatively low, especially for highly irregular problems. Inspired by the idea of adaptive finite element methods and incremental learning, we propose GAS, a Gaussian mixture distribution-based adaptive sampling method for PINNs. During the training procedure, GAS uses the current residual information to generate a Gaussian mixture distribution for the sampling of additional points, which are then trained together with historical data to speed up the convergence of the loss and achieve higher accuracy. Several numerical simulations on 2D and 10D problems show that GAS is a promising method that achieves state-of-the-art accuracy among deep solvers, while being comparable with traditional numerical solvers.

Figures (14)

• Figure 1: The flowchart of GAS method.
• Figure 2: PINNs with 1st adaptive sampling points in one peak problem and $|S_{\Omega}| = 1000$.
• Figure 3: PINNs with 5th adaptive sampling points in one peak problem and $|S_{\Omega}| = 3000$.
• Figure 4: PINNs with 9th adaptive sampling points in one peak problem and $|S_{\Omega}| = 5000$.
• Figure 5: Solutions and $3000$ points for uniform and GAS

Theorems & Definitions (4)

• Example 4.1
• Example 4.2
• Example 4.3
• Example 4.4