Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of gradient-dependent semilinear heat equations

Ariel Neufeld, Tuan Anh Nguyen

TL;DR

It is rigorously proved for the first time that deep neural networks can also overcome the curse dimensionality in the approximation of a certain class of nonlinear PDEs with gradient-dependent nonlinearities.

Abstract

Numerical experiments indicate that deep learning algorithms overcome the curse of dimensionality when approximating solutions of semilinear PDEs. For certain linear PDEs and semilinear PDEs with gradient-independent nonlinearities this has also been proved mathematically, i.e., it has been shown that the number of parameters of the approximating DNN increases at most polynomially in both the PDE dimension $d\in \mathbb{N}$ and the reciprocal of the prescribed accuracy $ε\in (0,1)$. The main contribution of this paper is to rigorously prove for the first time that deep neural networks can also overcome the curse dimensionality in the approximation of a certain class of nonlinear PDEs with gradient-dependent nonlinearities.

Rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of gradient-dependent semilinear heat equations

TL;DR

It is rigorously proved for the first time that deep neural networks can also overcome the curse dimensionality in the approximation of a certain class of nonlinear PDEs with gradient-dependent nonlinearities.

Abstract

Numerical experiments indicate that deep learning algorithms overcome the curse of dimensionality when approximating solutions of semilinear PDEs. For certain linear PDEs and semilinear PDEs with gradient-independent nonlinearities this has also been proved mathematically, i.e., it has been shown that the number of parameters of the approximating DNN increases at most polynomially in both the PDE dimension and the reciprocal of the prescribed accuracy . The main contribution of this paper is to rigorously prove for the first time that deep neural networks can also overcome the curse dimensionality in the approximation of a certain class of nonlinear PDEs with gradient-dependent nonlinearities.
Paper Structure (12 sections, 13 theorems, 124 equations)

This paper contains 12 sections, 13 theorems, 124 equations.

Table of Contents

  1. Introduction
  2. Notations
  3. A mathematical framework for DNNs
  4. Main result
  5. Outline of the proof and organization of the paper
  6. Deep neural networks
  7. Properties of operations associated to DNNs
  8. DNN representation of MLP approximations
  9. Perturbation lemma
  10. Proof of the main theorem
  11. Example
  12. Conclusion

Key Result

Theorem 1.2

Assume m07. Let $T\in (0,\infty)$, $\beta,c\in[2,\infty)$, $q\in [1,2)$. For every $d\in {\mathbbm{N}}$ let $L_i^d\in {\mathbbm{R}}$, $i\in [0,d]\cap{\mathbbm{Z}}$, satisfy that $\sum_{i=0}^{d}L_i^d\leq c$. For every $d\in {\mathbbm{N}}$ let $\Lambda^d=(\Lambda^d_{\nu})_{\nu\in [0,d]\cap{\mathbbm{Z} Then the following items hold.

Theorems & Definitions (21)

  • Theorem 1.2
  • Lemma 2.2: $\odot$ is associative--HJKN2020a
  • Lemma 2.3: $\boxplus$ and associativity--HJKN2020a
  • Lemma 2.4: Triangle inequality--HJKN2020a
  • Lemma 2.5
  • proof : Proof of \ref{['b15b']}
  • Lemma 2.6: DNNs for affine transformations--HJKN2020a
  • Lemma 2.7: DNNs for the multiplication with a vector
  • proof : Proof of \ref{['p01b']}
  • Lemma 2.8: Composition of functions generated by DNNs--HJKN2020a
  • ...and 11 more