Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

VS-PINN: A fast and efficient training of physics-informed neural networks using variable-scaling methods for solving PDEs with stiff behavior

Seungchan Ko, Sang Hyeon Park

TL;DR

The paper addresses the difficulty of training physics-informed neural networks on stiff and high-frequency PDEs by introducing VS-PINN, a simple variable-scaling approach that performs a scale transformation $x \mapsto \overline{x}/N$, rewrites the PDE accordingly, and minimizes a scaled loss to suppress stiffness while recovering the original solution via $u(x)=v(Nx)$. Through numerical experiments on the Wave, Allen–Cahn, boundary-layer, and Navier–Stokes equations, VS-PINN consistently improves training efficiency and accuracy without increasing data requirements. The authors support their empirical findings with Neural Tangent Kernel analysis, showing that scaling enhances the effective convergence rate by enlarging the kernel eigenvalues in the averaged metric. They also discuss practical guidelines for choosing the scaling factor $N$ and acknowledge limitations and future directions in theory and wider PDE applicability. Overall, VS-PINN provides a practical tool to mitigate spectral bias and stiffness in PINN training with broad potential impact for scientific computing workflows.

Abstract

Physics-informed neural networks (PINNs) have recently emerged as a promising way to compute the solutions of partial differential equations (PDEs) using deep neural networks. However, despite their significant success in various fields, it remains unclear in many aspects how to effectively train PINNs if the solutions of PDEs exhibit stiff behaviors or high frequencies. In this paper, we propose a new method for training PINNs using variable-scaling techniques. This method is simple and it can be applied to a wide range of problems including PDEs with rapidly-varying solutions. Throughout various numerical experiments, we will demonstrate the effectiveness of the proposed method for these problems and confirm that it can significantly improve the training efficiency and performance of PINNs. Furthermore, based on the analysis of the neural tangent kernel (NTK), we will provide theoretical evidence for this phenomenon and show that our methods can indeed improve the performance of PINNs.

VS-PINN: A fast and efficient training of physics-informed neural networks using variable-scaling methods for solving PDEs with stiff behavior

TL;DR

The paper addresses the difficulty of training physics-informed neural networks on stiff and high-frequency PDEs by introducing VS-PINN, a simple variable-scaling approach that performs a scale transformation , rewrites the PDE accordingly, and minimizes a scaled loss to suppress stiffness while recovering the original solution via . Through numerical experiments on the Wave, Allen–Cahn, boundary-layer, and Navier–Stokes equations, VS-PINN consistently improves training efficiency and accuracy without increasing data requirements. The authors support their empirical findings with Neural Tangent Kernel analysis, showing that scaling enhances the effective convergence rate by enlarging the kernel eigenvalues in the averaged metric. They also discuss practical guidelines for choosing the scaling factor and acknowledge limitations and future directions in theory and wider PDE applicability. Overall, VS-PINN provides a practical tool to mitigate spectral bias and stiffness in PINN training with broad potential impact for scientific computing workflows.

Abstract

Physics-informed neural networks (PINNs) have recently emerged as a promising way to compute the solutions of partial differential equations (PDEs) using deep neural networks. However, despite their significant success in various fields, it remains unclear in many aspects how to effectively train PINNs if the solutions of PDEs exhibit stiff behaviors or high frequencies. In this paper, we propose a new method for training PINNs using variable-scaling techniques. This method is simple and it can be applied to a wide range of problems including PDEs with rapidly-varying solutions. Throughout various numerical experiments, we will demonstrate the effectiveness of the proposed method for these problems and confirm that it can significantly improve the training efficiency and performance of PINNs. Furthermore, based on the analysis of the neural tangent kernel (NTK), we will provide theoretical evidence for this phenomenon and show that our methods can indeed improve the performance of PINNs.
Paper Structure (15 sections, 2 theorems, 63 equations, 14 figures)

This paper contains 15 sections, 2 theorems, 63 equations, 14 figures.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Physics-Informed Neural Networks
  4. Neural Tangent Kernel
  5. Neural Tangent Kernel for PINNs
  6. Variable-Scaling Physics-Informed Neural Networks
  7. Numerical Experiments
  8. Wave Equation
  9. Allen--Cahn Equation
  10. Boundary-Layer Problem
  11. The Navier--Stokes Equations
  12. Neural Tangent Kernel Analysis
  13. Averaged Convergence Rate
  14. Proper choice of the scaling factor $N$
  15. Conclusion

Key Result

Theorem 2.1

Assume that the activation function $\sigma$ is of class $C^2$ and Lipschitz, and the loss function $\mathcal{L}$ is Lipschitz. Then the neural tangent kernel $\widehat{\boldsymbol{\Theta}}(t)$ converges in probability to a deterministic kernel independent of $t>0$ as $n_1$, $n_2,\cdots n_{L-1}\righ

Figures (14)

  • Figure 1: Schematic illustration of the physics-informed neural network. The left part visualizes a standard feed-forward neural network parameterized by $\theta$, while the right part imposes the given physical laws to the neural network.
  • Figure 2: An illustration of the effect of the variable scaling when the scaling factor $N=50$ is applied to the function $\exp( -10000x^2)$. The scaling has the effect of "zoom-in" and the resulting function exhibits relatively less stiff behavior on the designated domain.
  • Figure 3: Learning curves for the VS-PINNs with the scales $N=1,4,10$.
  • Figure 4: One-dimensional wave equation: (a) The exact solution versus the predicted solution by training the standard PINN. The relative $L^2$ error is 6.31e-1. (b) The exact solution versus the predicted solution by training the VS-PINN with the scale $N=10$. The relative $L^2$ error is 1.20e-2.
  • Figure 5: Traning dynamics for the Allen--Cahn equation using VS-PINN with the scales $N=1$ (standard PINN), $10$, $100$. (a) is for the model trained with the pre-computed training data at $t=0.1$ and (b) is for the model trained in an unsupervised manner.
  • ...and 9 more figures

Theorems & Definitions (2)

  • Theorem 2.1
  • Lemma 5.1