Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Networks in Numerical Analysis and Approximation Theory

Gonzalo Romera

TL;DR

A Neural Network is built that approximates the inverse of positive-definite symmetric matrices, which allows to get a Garlerkin numerical solution of elliptic PDEs.

Abstract

In this Master Thesis, we study the approximation capabilities of Neural Networks in the context of numerical resolution of elliptic PDEs and Approximation Theory. First of all, in Chapter 1, we introduce the mathematical definition of Neural Networks and perform some basic estimates on their composition and parallelization. Then, we implement in Chapter 2 the Galerkin method using Neural Network. In particular, we manage to build a Neural Network that approximates the inverse of positive-definite symmetric matrices, which allows to get a Garlerkin numerical solution of elliptic PDEs. Finally, in Chapter 3, we introduce the approximation space of Neural Networks, a space which consists of functions in $L^p$ that are approximated at a certain rate when increasing the number of weights of Neural Networks. We find the relation of this space with the Besov space: the smoother a function is, the faster it can be approximated with Neural Networks when increasing the number of weights.

Neural Networks in Numerical Analysis and Approximation Theory

TL;DR

A Neural Network is built that approximates the inverse of positive-definite symmetric matrices, which allows to get a Garlerkin numerical solution of elliptic PDEs.

Abstract

In this Master Thesis, we study the approximation capabilities of Neural Networks in the context of numerical resolution of elliptic PDEs and Approximation Theory. First of all, in Chapter 1, we introduce the mathematical definition of Neural Networks and perform some basic estimates on their composition and parallelization. Then, we implement in Chapter 2 the Galerkin method using Neural Network. In particular, we manage to build a Neural Network that approximates the inverse of positive-definite symmetric matrices, which allows to get a Garlerkin numerical solution of elliptic PDEs. Finally, in Chapter 3, we introduce the approximation space of Neural Networks, a space which consists of functions in that are approximated at a certain rate when increasing the number of weights of Neural Networks. We find the relation of this space with the Besov space: the smoother a function is, the faster it can be approximated with Neural Networks when increasing the number of weights.
Paper Structure (21 sections, 27 theorems, 329 equations, 3 figures)

This paper contains 21 sections, 27 theorems, 329 equations, 3 figures.

Table of Contents

  1. Introduction to Neural Networks
  2. Neural Network Definition and its Parameters.
  3. Basic Results.
  4. Additional Definitions of Neural Network in the Literature.
  5. Deep Neural Networks for the numerical resolution of PDEs
  6. Galerkin Method.
  7. Neural Network construction.
  8. Product Neural Network.
  9. Matrix Product Neural Network.
  10. Matrix Inversion Neural Network.
  11. Other Ways of Solving PDEs.
  12. Approximation Space of Neural Networks
  13. Introductory Notions.
  14. Approximation Spaces.
  15. Besov Spaces.
  16. ...and 6 more sections

Key Result

Lemma 1.1.3

Let $A\in\mathbb{R}^{d\times d}$, $B\in\mathbb{R}^{d\times l}$ be matrices and $v\in\mathbb{R}^{1\times n}$ a row vector. Then and if every row of $B$ has at most one non-zero entry then

Figures (3)

  • Figure 1: Plot of $g^m$ for $m = 1,2,3,4$.
  • Figure 2: Plot of $h_m$ for $m = 0, 1, 2, 3$.
  • Figure 3: NN scheme of $\Phi_{\cdot,k+1}$.

Theorems & Definitions (80)

  • Definition 1.1.1: Neural Network
  • Definition 1.1.2
  • Lemma 1.1.3
  • Definition 1.1.4
  • Definition 1.1.5: Realization
  • Definition 1.2.1
  • Definition 1.2.2
  • Definition 1.2.3
  • Definition 1.2.4
  • Lemma 1.2.5
  • ...and 70 more