Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Physics-informed deep learning and compressive collocation for high-dimensional diffusion-reaction equations: practical existence theory and numerics

Simone Brugiapaglia, Nick Dexter, Samir Karam, Weiqi Wang

TL;DR

This work investigates solving high-dimensional diffusion-reaction PDEs using two complementary approaches: compressive Fourier collocation (CFC) and physics-informed neural networks (PINNs) equipped with a periodic layer. It develops a practical existence theorem (PET) showing that trainable periodic PINNs can achieve accuracy comparable to sparse Fourier approximations with sample complexity that scales only logarithmically or linearly in the dimension, under sparsity assumptions on the Fourier representation and mild coefficient conditions. The paper extends CFC theory to diffusion-reaction problems, introduces adaptive lower OMP to improve high-dimensional recovery, and provides extensive numerical comparisons up to dimension $d=30$, illustrating the trade-offs: CFC achieves higher accuracy for sparse solutions, while periodic PINNs offer robust, parameter-light performance across varied tests. Together, the theoretical results and numerical experiments illuminate the potential and limitations of combining DL-based solvers with sparse spectral methods for scalable, high-dimensional PDEs, and outline open directions for time-dependent problems and non-periodic domains. The work thus contributes a rigorous bridge between practical neural-network solvers and sparse approximation theory in the context of high-dimensional PDEs.

Abstract

On the forefront of scientific computing, Deep Learning (DL), i.e., machine learning with Deep Neural Networks (DNNs), has emerged a powerful new tool for solving Partial Differential Equations (PDEs). It has been observed that DNNs are particularly well suited to weakening the effect of the curse of dimensionality, a term coined by Richard E. Bellman in the late `50s to describe challenges such as the exponential dependence of the sample complexity, i.e., the number of samples required to solve an approximation problem, on the dimension of the ambient space. However, although DNNs have been used to solve PDEs since the `90s, the literature underpinning their mathematical efficiency in terms of numerical analysis (i.e., stability, accuracy, and sample complexity), is only recently beginning to emerge. In this paper, we leverage recent advancements in function approximation using sparsity-based techniques and random sampling to develop and analyze an efficient high-dimensional PDE solver based on DL. We show, both theoretically and numerically, that it can compete with a novel stable and accurate compressive spectral collocation method for the solution of high-dimensional, steady-state diffusion-reaction equations with periodic boundary conditions. In particular, we demonstrate a new practical existence theorem, which establishes the existence of a class of trainable DNNs with suitable bounds on the network architecture and a sufficient condition on the sample complexity, with logarithmic or, at worst, linear scaling in dimension, such that the resulting networks stably and accurately approximate a diffusion-reaction PDE with high probability.

Physics-informed deep learning and compressive collocation for high-dimensional diffusion-reaction equations: practical existence theory and numerics

TL;DR

This work investigates solving high-dimensional diffusion-reaction PDEs using two complementary approaches: compressive Fourier collocation (CFC) and physics-informed neural networks (PINNs) equipped with a periodic layer. It develops a practical existence theorem (PET) showing that trainable periodic PINNs can achieve accuracy comparable to sparse Fourier approximations with sample complexity that scales only logarithmically or linearly in the dimension, under sparsity assumptions on the Fourier representation and mild coefficient conditions. The paper extends CFC theory to diffusion-reaction problems, introduces adaptive lower OMP to improve high-dimensional recovery, and provides extensive numerical comparisons up to dimension , illustrating the trade-offs: CFC achieves higher accuracy for sparse solutions, while periodic PINNs offer robust, parameter-light performance across varied tests. Together, the theoretical results and numerical experiments illuminate the potential and limitations of combining DL-based solvers with sparse spectral methods for scalable, high-dimensional PDEs, and outline open directions for time-dependent problems and non-periodic domains. The work thus contributes a rigorous bridge between practical neural-network solvers and sparse approximation theory in the context of high-dimensional PDEs.

Abstract

On the forefront of scientific computing, Deep Learning (DL), i.e., machine learning with Deep Neural Networks (DNNs), has emerged a powerful new tool for solving Partial Differential Equations (PDEs). It has been observed that DNNs are particularly well suited to weakening the effect of the curse of dimensionality, a term coined by Richard E. Bellman in the late `50s to describe challenges such as the exponential dependence of the sample complexity, i.e., the number of samples required to solve an approximation problem, on the dimension of the ambient space. However, although DNNs have been used to solve PDEs since the `90s, the literature underpinning their mathematical efficiency in terms of numerical analysis (i.e., stability, accuracy, and sample complexity), is only recently beginning to emerge. In this paper, we leverage recent advancements in function approximation using sparsity-based techniques and random sampling to develop and analyze an efficient high-dimensional PDE solver based on DL. We show, both theoretically and numerically, that it can compete with a novel stable and accurate compressive spectral collocation method for the solution of high-dimensional, steady-state diffusion-reaction equations with periodic boundary conditions. In particular, we demonstrate a new practical existence theorem, which establishes the existence of a class of trainable DNNs with suitable bounds on the network architecture and a sufficient condition on the sample complexity, with logarithmic or, at worst, linear scaling in dimension, such that the resulting networks stably and accurately approximate a diffusion-reaction PDE with high probability.
Paper Structure (51 sections, 5 theorems, 118 equations, 13 figures, 1 algorithm)

This paper contains 51 sections, 5 theorems, 118 equations, 13 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Main contributions
  3. i) New convergence theorem for high-dimensional periodic PINNs.
  4. ii) Numerical study of CFC and periodic PINNs in high dimensions.
  5. iii) Improved implementation of CFC.
  6. iv) CFC convergence theorem for diffusion-reaction problems.
  7. Contextualization of our contributions
  8. Outline of the paper
  9. Problem setting
  10. Notation.
  11. Model problem.
  12. Physics-Informed Neural Networks (PINNs) with periodic layer
  13. A practical existence theorem for periodic PINNs
  14. Target accuracy: sparse Fourier approximation.
  15. Sufficient conditions on the PDE coefficients.
  16. ...and 36 more sections

Key Result

Theorem 1

Given a dimension $d\in \mathbb{N}$, target sparsity $s\in\mathbb{N}$, hyperbolic cross order $n\in \mathbb{N}$, RePU power $\ell \in \mathbb{N}$, with $\ell \geq 2$, and probability of failure $\varepsilon \in (0,1)$, there exist: such that the following holds with probability $1-\varepsilon$. For all $a:\mathbb{T}^d \to \mathbb{R}$ and $\rho \in \mathbb{R}$ satisfying eq:suff_cond_a_rho_f, eq:d

Figures (13)

  • Figure 1: Architecture of the neural network with the periodic layer ($d=2$). The number in superscript represents the layer number. In this case, $h = 3$ and depth$(\psi) = 7$.
  • Figure 2: Illustration of the greedy selection criterion of adaptive lower OMP (Line \ref{['step:greedy']} in Algorithm \ref{['algo:lower_OMP']}). Considering a given lower set $\Lambda^{(n)} \subset \mathbb{Z}^2$ (blue), its reduced margin $\mathcal{R}(\Lambda^{(n)})$ is drawn in red. If the quantity $|A^*(\bm{b}-A\bm{z}^{(n)})_{\bm{\nu}}|$ is minimized at $\bm{\nu} = (3,1)$, then $\Lambda^{(n+1)}$ is constructed by adding the red circled dots to $\Lambda^{(n)}$.
  • Figure 3: (Impact of the number of samples) Relative $L^2$-error versus the number of epochs for approximating the exact solutions defined in \ref{['eq:exact1']}-\ref{['eq:exact3']} with $d=6$, where $m$ is the number of sample points.
  • Figure 4: (Impact of the dimension) Relative $L^2$-error versus the number of epochs for approximating the exact solutions defined in \ref{['eq:exact1']}-\ref{['eq:exact3']} with (top left, bottom left, bottom right) $m=3000$ and (top right) $m=10000$ samples, in $d = 6, 10, 20$ dimensions.
  • Figure 5: (Dimensionality and number of samples) Relative $L^2$-error after 30000 epochs versus number of sample points $m$ for the exact solutions defined in Eq. \ref{['eq:exact1']}-\ref{['eq:exact3']}, where $d= 6, 10, 20$ are the dimension of the problem. Left: Example 1. Middle: Example 2. Right: Example 3.
  • ...and 8 more figures

Theorems & Definitions (10)

  • Theorem 1: Practical existence theorem for high-dimensional periodic PINNs
  • Definition 2: Lower set of $\mathbb{Z}^d$
  • Definition 3: Reduced margin
  • Definition 4: Bounded Riesz System
  • Lemma 5: Gershgorin's circle theorem for Hermitian matrices
  • Lemma 6: Explicit formula for the Gram matrix
  • Lemma 7
  • Theorem 8: Convergence of CFC for diffusion-reaction problems
  • Remark 9
  • Remark 10