Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Number Theoretic Accelerated Learning of Physics-Informed Neural Networks

Takashi Matsubara, Takaharu Yaguchi

TL;DR

This work tackles the computational bottleneck of physics-informed neural networks by addressing the discretization error introduced by finite collocation points in the physics-informed loss. It introduces Good Lattice Training (GLT), a lattice-based collocation scheme rooted in Korobov space theory, with periodization and randomization tricks that yield an improved quadrature error bound of $O\left(\frac{(\log N)^{\alpha s}}{N^{\alpha}}\right)$ under smoothness $\alpha$. Empirically, GLT achieves 2–7× reductions in required collocation points while maintaining competitive accuracy across PDEs (nonlinear Schrödinger, KdV, Allen–Cahn, Poisson) and PINN variants (CPINN), and it enhances parameter identification in system identification tasks. The method shows clear computational savings in low-dimensional settings (s ≤ 4) and suggests a practical approach to scaling PINNs with reduced training cost, though higher-dimensional cases reveal more nuanced behavior and potential limitations.

Abstract

Physics-informed neural networks solve partial differential equations by training neural networks. Since this method approximates infinite-dimensional PDE solutions with finite collocation points, minimizing discretization errors by selecting suitable points is essential for accelerating the learning process. Inspired by number theoretic methods for numerical analysis, we introduce good lattice training and periodization tricks, which ensure the conditions required by the theory. Our experiments demonstrate that GLT requires 2-7 times fewer collocation points, resulting in lower computational cost, while achieving competitive performance compared to typical sampling methods.

Number Theoretic Accelerated Learning of Physics-Informed Neural Networks

TL;DR

This work tackles the computational bottleneck of physics-informed neural networks by addressing the discretization error introduced by finite collocation points in the physics-informed loss. It introduces Good Lattice Training (GLT), a lattice-based collocation scheme rooted in Korobov space theory, with periodization and randomization tricks that yield an improved quadrature error bound of under smoothness . Empirically, GLT achieves 2–7× reductions in required collocation points while maintaining competitive accuracy across PDEs (nonlinear Schrödinger, KdV, Allen–Cahn, Poisson) and PINN variants (CPINN), and it enhances parameter identification in system identification tasks. The method shows clear computational savings in low-dimensional settings (s ≤ 4) and suggests a practical approach to scaling PINNs with reduced training cost, though higher-dimensional cases reveal more nuanced behavior and potential limitations.

Abstract

Physics-informed neural networks solve partial differential equations by training neural networks. Since this method approximates infinite-dimensional PDE solutions with finite collocation points, minimizing discretization errors by selecting suitable points is essential for accelerating the learning process. Inspired by number theoretic methods for numerical analysis, we introduce good lattice training and periodization tricks, which ensure the conditions required by the theory. Our experiments demonstrate that GLT requires 2-7 times fewer collocation points, resulting in lower computational cost, while achieving competitive performance compared to typical sampling methods.
Paper Structure (31 sections, 5 theorems, 41 equations, 7 figures, 2 tables)

This paper contains 31 sections, 5 theorems, 41 equations, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Theoretical Error Estimate of PINNs
  5. Good Lattice Training
  6. Periodization and Randomization Tricks
  7. Limitations
  8. Experiments and Results
  9. Physics-Informed Neural Networks
  10. Results: Accuracy of Physics-Informed Loss
  11. Results: Performance of PINNs
  12. Results: System Identification
  13. Competitive Physics-Informed Neural Networks
  14. Results
  15. Conclusion
  16. ...and 16 more sections

Key Result

Theorem 1

Suppose that the class of neural networks used for PINNs includes an $\varepsilon_1$-approximator $\tilde{u}_\mathrm{opt}$ to the exact solution $u^*$ to the PDE $\mathcal{N}[u]=0$: $\|u^* - \tilde{u}_\mathrm{opt} \| \leq \varepsilon_1$, and that (eq:dloss) is an $\varepsilon_2$-approximation of $(e

Figures (7)

  • Figure 1: Examples of sampled collocation points. 128 points for the Sobol sequence, and 144 points for others.
  • Figure 2: The number $N$ of collocation points and the standard deviation of the physics-informed loss, which approximates the discretization error $|(\ref{['eq:closs']})-(\ref{['eq:dloss']})|$.
  • Figure 3: The results of PINNs. The number $N$ of collocation points and the relative error $\mathcal{L}$.
  • Figure 5: The results of CPINNs. The number of iterations and the relative error $\mathcal{L}$.
  • Figure A1: Example results at competitive number $N$ of collocation points (on vertical green line in Fig. \ref{['fig:PINNsResults']}). The leftmost panel shows the true solution. The remaining panels show the residuals of PINNs' results using uniformly random sampling, uniformly spaced sampling, LHS, Sobol sequence, and GLT, from left to right. The residuals are multiplied by the factors in parentheses.
  • ...and 2 more figures

Theorems & Definitions (9)

  • Theorem 1
  • Definition 2: Sloan1994-cl
  • Definition 3: Zaremba1972-gj
  • Lemma 4: Zaremba1972-gj
  • Definition 5: Zaremba1972-gj
  • Theorem 6: Sloan1994-cl
  • Theorem 7
  • Theorem
  • proof