Is the neural tangent kernel of PINNs deep learning general partial differential equations always convergent ?

TL;DR

This work analyzes the neural tangent kernel (NTK) for physics-informed neural networks (PINNs) solving general PDEs, linking training dynamics to NTK evolution and highlighting the role of operator homogeneity in convergence. It establishes that the initialized NTK converges in probability to a deterministic kernel \\mathbf{K}^* as network width \\N \ ightarrow \\infty, and that the NTK remains effectively constant during training for sufficiently large scaling exponent \\textit{s} (\\lim_{N\\to\\infty} \\sup_{t\\in[0,T]} \\|\\mathbf{K}(t)-\\mathbf{K}(0)\\|_2=0 with \\textit{s} > 1/4). The paper proves these results (Theorems 2.2 and 2.3) and illustrates them with two PDE examples (sine-Gordon IVP and KdV IBVP), showing that convergence hinges on the homogeneity of nonlinear terms and the chosen initialization scale. The findings provide actionable guidelines for initializing PINNs to ensure stable NTK behavior and reliable training when solving general PDEs. Overall, the work broadens NTK applicability in PINNs and clarifies conditions under which NTK-based analyses remain valid.

Abstract

In this paper, we study the neural tangent kernel (NTK) for general partial differential equations (PDEs) based on physics-informed neural networks (PINNs). As we all know, the training of an artificial neural network can be converted to the evolution of NTK. We analyze the initialization of NTK and the convergence conditions of NTK during training for general PDEs. The theoretical results show that the homogeneity of differential operators plays a crucial role for the convergence of NTK. Moreover, based on the PINNs, we validate the convergence conditions of NTK using the initial value problems of the sine-Gordon equation and the initial-boundary value problem of the KdV equation.

Figures (5)

• Figure 1: The trends of $\|\mathbf{K}_{N+k}-\mathbf{K}_N\|_2$ of the initial value problem of the sine-Gordon equation . The blue area is the variation range of the results of 50 independent experiments, and the dark line is the mean of these experiments.
• Figure 2: The trends of $\|\mathbf{K}_{N+k}-\mathbf{K}_N\|_2$ of the initial value problem of the sine-Gordon equation (\ref{['IPsG']}) when (a) $s=0.5$, (b) $s=1$. The blue area is the variation range of the results of 50 independent experiments, and the black line denotes the mean value of these experiments.
• Figure 3: The trends of $\sup\limits_{t\in[0,n]}\|\mathbf{K}_N(t)-\mathbf{K}_N(0)\|_2$ of the initial value problem of the sine-Gordon equation (\ref{['IPsG']}) when $s=1/4$, where $n$ represents the maximal training step and $N$ the width of the neural networks.
• Figure 4: The trends of $\|\mathbf{K}_{N+k}-\mathbf{K}_N\|_2$ of the initial value problem of the KdV equation (\ref{['IBPKdV']}) when (a) $s=0.5$, (b) $s=1$. The blue area is the variation range of the results of 50 independent experiments, and the black line is the mean value of these experiments.
• Figure 5: The trends of $\sup\limits_{t\in[0,n]}\|\mathbf{K}_N(t)-\mathbf{K}_N(0)\|_2$ of the initial value problem of the KdV equation when $s=0.2$, where $n$ represents the maximal train step and $N$ the width of the neural networks.