Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Complexity bounds on neural networks for the solution of structured linear systems of equations

Benjamin Dörich, Roland Maier, Lukas Ullmer

Abstract

We derive upper bounds on the complexity of ReLU neural networks approximating the solution of a linear system given the matrix and the right-hand side. We focus on matrices which are symmetric positive definite and sparse, as they appear in the context of finite difference and finite element methods. For such matrices, we extend available results for the matrix inversion to the task of solving a linear system, where we leverage favorable properties of classical methods such as the modified Richardson and the conjugate gradient method. Our bounds on the number of layers and neurons are not only explicit with respect to the size of the matrices, but also with respect to their condition numbers.

Complexity bounds on neural networks for the solution of structured linear systems of equations

Abstract

We derive upper bounds on the complexity of ReLU neural networks approximating the solution of a linear system given the matrix and the right-hand side. We focus on matrices which are symmetric positive definite and sparse, as they appear in the context of finite difference and finite element methods. For such matrices, we extend available results for the matrix inversion to the task of solving a linear system, where we leverage favorable properties of classical methods such as the modified Richardson and the conjugate gradient method. Our bounds on the number of layers and neurons are not only explicit with respect to the size of the matrices, but also with respect to their condition numbers.
Paper Structure (9 sections, 13 theorems, 80 equations)

This paper contains 9 sections, 13 theorems, 80 equations.

Table of Contents

  1. Introduction
  2. Notation
  3. Main results
  4. Abstract results
  5. Application to finite element methods
  6. Preliminaries for the proofs
  7. Richardson method - proof of Theorem \ref{['thm:Richardson']}
  8. Cg-type method - proof of Theorem \ref{['thm:cg']}
  9. Possible extensions

Key Result

Theorem 2.3

Let $\delta \in (0,1)$. Then, for any $\epsilon\in(0,\frac{1}{4})$ there exists a neural network $\Phi^{\textup{inv}}$ such that where $\mathrm{dim}_{\mathrm{in}}(\Phi^{\textup{inv}})=n^2$ and $\mathrm{dim}_{\mathrm{out}}(\Phi^{\textup{inv}})=n^2$. Further, with there exists a constant $C_\textup{inv} >0$ which is independent of $m_{\textup{N}}$, $n$, $\epsilon$, and $\delta$ such that

Theorems & Definitions (30)

  • Remark 2.1
  • Definition 2.2
  • Theorem 2.3: cf. KutPRS22
  • Theorem 3.1: Approximation via modified Richardson iteration
  • proof
  • Theorem 3.2: Approximation via cg-type iteration
  • proof
  • Remark 3.3
  • Remark 3.4
  • Remark 3.5
  • ...and 20 more