Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Non-Asymptotic Analysis of Projected Gradient Descent for Physics-Informed Neural Networks

Jonas Nießen, Johannes Müller

TL;DR

This paper develops a non-asymptotic analysis of PINNs trained with projected gradient descent for the Poisson equation. By relating the training dynamics to a linearized neural tangent kernel (NTK) framework and employing a Lyapunov-drift argument, it proves a convergence bound of $O(1/\sqrt{T} + 1/\sqrt{m} + \epsilon_{\mathrm{approx}})$ without requiring strong over-parameterization. It also bounds the generalization error via the Rademacher complexities of the network and its Laplacian, and combines these with regularity theory to derive an overall $H^{1/2}(\Omega)$ error bound. The results provide theoretical guarantees for under-parameterized PINNs and guidance on choosing network width, iteration count, and approximation radius in practice.

Abstract

In this work, we provide a non-asymptotic convergence analysis of projected gradient descent for physics-informed neural networks for the Poisson equation. Under suitable assumptions, we show that the optimization error can be bounded by $\mathcal{O}(1/\sqrt{T} + 1/\sqrt{m} + ε_{\text{approx}})$, where $T$ is the number of algorithm time steps, $m$ is the width of the neural network and $ε_{\text{approx}}$ is an approximation error. The proof of our optimization result relies on bounding the linearization error and using this result together with a Lyapunov drift analysis. Additionally, we quantify the generalization error by bounding the Rademacher complexities of the neural network and its Laplacian. Combining both the optimization and generalization results, we obtain an overall error estimate based on an existing error estimate from regularity theory.

Non-Asymptotic Analysis of Projected Gradient Descent for Physics-Informed Neural Networks

TL;DR

This paper develops a non-asymptotic analysis of PINNs trained with projected gradient descent for the Poisson equation. By relating the training dynamics to a linearized neural tangent kernel (NTK) framework and employing a Lyapunov-drift argument, it proves a convergence bound of without requiring strong over-parameterization. It also bounds the generalization error via the Rademacher complexities of the network and its Laplacian, and combines these with regularity theory to derive an overall error bound. The results provide theoretical guarantees for under-parameterized PINNs and guidance on choosing network width, iteration count, and approximation radius in practice.

Abstract

In this work, we provide a non-asymptotic convergence analysis of projected gradient descent for physics-informed neural networks for the Poisson equation. Under suitable assumptions, we show that the optimization error can be bounded by , where is the number of algorithm time steps, is the width of the neural network and is an approximation error. The proof of our optimization result relies on bounding the linearization error and using this result together with a Lyapunov drift analysis. Additionally, we quantify the generalization error by bounding the Rademacher complexities of the neural network and its Laplacian. Combining both the optimization and generalization results, we obtain an overall error estimate based on an existing error estimate from regularity theory.
Paper Structure (8 sections, 9 theorems, 69 equations, 1 figure, 1 algorithm)

This paper contains 8 sections, 9 theorems, 69 equations, 1 figure, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries on Physics-Informed Neural Networks
  3. Optimization Guarantee for Projected GD for PINNs
  4. Neural Tangent Kernels and Reproducing Kernel Hilbert Spaces
  5. Linearization and Approximation Errors
  6. Finite-Time Bound for Projected Gradient Descent
  7. A Computational Example
  8. Overall Error Analysis

Key Result

Lemma 4

Let $m\in \mathbb{N}_e$, fix $\theta(0)$ and let $\theta_i\in {B_2}(\theta_i(0), \frac{p}{\sqrt{m}})$ for all $i\in\{1, \ldots, m\}$, then there exists a constant $C = C(\sigma_2, \sigma_3, \sigma_4)>0$ such that for all $x\in \mathbb{R}^d$ we have

Figures (1)

  • Figure 1: Shown are the $90\%$ percentile of the empirical loss compared to different network widths $m$ trained with projected stochastic gradient descent for $T=m$ steps as well as the $\mathcal{O}\bigl(\frac{1}{\sqrt{T}}\bigr) = \mathcal{O}\bigl(\frac{1}{\sqrt{m}}\bigr)$ rate, which is guaranteed by \ref{['thm:optimization bound']} up to an approximation error.

Theorems & Definitions (22)

  • Remark 1
  • Remark 3
  • Lemma 4: Linearization error
  • Lemma 5: Approximation error
  • Theorem 6: Performance of Projected Gradient Descent
  • Remark 7: Realizability
  • proof : Proof of \ref{['thm:optimization bound']}
  • Theorem 8: Complete error analysis
  • Proposition 9: Estimate on Rademacher complexity
  • proof
  • ...and 12 more