Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SGM-PINN: Sampling Graphical Models for Faster Training of Physics-Informed Neural Networks

John Anticev, Ali Aghdaei, Wuxinlin Cheng, Zhuo Feng

TL;DR

This paper addresses the slow and sometimes unreliable convergence of physics-informed neural networks (PINNs) when solving parameterized PDEs. It introduces SGM-PINN, a graph-based importance sampling framework that builds a probabilistic graphical model from the training point cloud, uses low-resistance-diameter (LRD) spectral clustering to form highly informative sample clusters, and prioritizes sampling from these clusters to reduce batch sizes and overall training time. A stability scoring mechanism based on inverse stability rating (ISR) is integrated to incorporate gradient- and parameter-variation information, improving robustness for parameterized PINNs. The approach yields 2–3x faster convergence on large CFD problems with maintained or improved solution accuracy and demonstrates practical potential for scalable, fast PDE surrogates in design and optimization tasks.

Abstract

SGM-PINN is a graph-based importance sampling framework to improve the training efficacy of Physics-Informed Neural Networks (PINNs) on parameterized problems. By applying a graph decomposition scheme to an undirected Probabilistic Graphical Model (PGM) built from the training dataset, our method generates node clusters encoding conditional dependence between training samples. Biasing sampling towards more important clusters allows smaller mini-batches and training datasets, improving training speed and accuracy. We additionally fuse an efficient robustness metric with residual losses to determine regions requiring additional sampling. Experiments demonstrate the advantages of the proposed framework, achieving $3\times$ faster convergence compared to prior state-of-the-art sampling methods.

SGM-PINN: Sampling Graphical Models for Faster Training of Physics-Informed Neural Networks

TL;DR

This paper addresses the slow and sometimes unreliable convergence of physics-informed neural networks (PINNs) when solving parameterized PDEs. It introduces SGM-PINN, a graph-based importance sampling framework that builds a probabilistic graphical model from the training point cloud, uses low-resistance-diameter (LRD) spectral clustering to form highly informative sample clusters, and prioritizes sampling from these clusters to reduce batch sizes and overall training time. A stability scoring mechanism based on inverse stability rating (ISR) is integrated to incorporate gradient- and parameter-variation information, improving robustness for parameterized PINNs. The approach yields 2–3x faster convergence on large CFD problems with maintained or improved solution accuracy and demonstrates practical potential for scalable, fast PDE surrogates in design and optimization tasks.

Abstract

SGM-PINN is a graph-based importance sampling framework to improve the training efficacy of Physics-Informed Neural Networks (PINNs) on parameterized problems. By applying a graph decomposition scheme to an undirected Probabilistic Graphical Model (PGM) built from the training dataset, our method generates node clusters encoding conditional dependence between training samples. Biasing sampling towards more important clusters allows smaller mini-batches and training datasets, improving training speed and accuracy. We additionally fuse an efficient robustness metric with residual losses to determine regions requiring additional sampling. Experiments demonstrate the advantages of the proposed framework, achieving faster convergence compared to prior state-of-the-art sampling methods.
Paper Structure (22 sections, 3 theorems, 12 equations, 4 figures, 2 tables, 1 algorithm)

This paper contains 22 sections, 3 theorems, 12 equations, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Theoretical background of PINNs.
  4. Importance Sampling for Training PINNs
  5. Current State of the Art.
  6. Limitations of existing IS methods.
  7. The Proposed SGM-PINN Framework
  8. Overview
  9. The proposed SGM-PINN framework
  10. Advantages of our approach
  11. Constructing PGMs from Point Cloud (S1)
  12. Node Clustering by LRD Decomposition (S2)
  13. Stability Score for Parameterized PINNs (S3)
  14. Batch Ranking and Selection (S4)
  15. Algorithm Flow and Complexity
  16. ...and 7 more sections

Key Result

Lemma 1

Let $H^{(t)} = \nabla_\theta\mathcal{L}(\theta^{(t)})$ the gradient in Equation eqn:sgd_update_rule. The convergence of SGD can be accelerated by sampling the input variables from a distribution $P$ that minimizes $\mathbf{Trace}(\mathbb{V}_P[H^{(t)}])$, which is accomplished when the probability $P

Figures (4)

  • Figure 1: The SGM-PINN framework for training PINNs.
  • Figure 2: Solution error by wall time (lower) for $v$ in the LDC example. $Uniform_{500}$, $SGM$-$PINN_{500}$, and $MIS_{500}$ have batch sizes of 500 and a total of 500,000 collocation points. The baseline $Uniform_{4000}$ has a batch size of 4000 and 4M total collocation points.
  • Figure 3: Solution errors of $v$ for parameterized PINN for the AR example compared to the OpenFOAM validation data averaged at $r_i$=1.0,0.88, and 0.75, respectively.
  • Figure 4: Visualized absolute errors for $p$ at $r_i$=1.0

Theorems & Definitions (4)

  • Lemma 1
  • Definition 3.1
  • Lemma 2
  • Lemma 3