Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Networks are Integrable

Yucong Liu

TL;DR

This study introduces a novel numerical method that consist of a forward algorithm and a corrective procedure for the integration of multi-layer Neural Networks, a challenging task within this domain.

Abstract

In this study, we explore the integration of Neural Networks, a powerful class of functions known for their exceptional approximation capabilities. Our primary emphasis is on the integration of multi-layer Neural Networks, a challenging task within this domain. To tackle this challenge, we introduce a novel numerical method that consist of a forward algorithm and a corrective procedure. Our experimental results demonstrate the accuracy achieved through our integration approach.

Neural Networks are Integrable

TL;DR

This study introduces a novel numerical method that consist of a forward algorithm and a corrective procedure for the integration of multi-layer Neural Networks, a challenging task within this domain.

Abstract

In this study, we explore the integration of Neural Networks, a powerful class of functions known for their exceptional approximation capabilities. Our primary emphasis is on the integration of multi-layer Neural Networks, a challenging task within this domain. To tackle this challenge, we introduce a novel numerical method that consist of a forward algorithm and a corrective procedure. Our experimental results demonstrate the accuracy achieved through our integration approach.
Paper Structure (12 sections, 2 theorems, 6 equations, 2 figures, 2 algorithms)

This paper contains 12 sections, 2 theorems, 6 equations, 2 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Integral and Algorithms
  3. Basic Motivation
  4. Integral over one-layer Neural Networks
  5. Numerical Integral over multi-layer Neural Networks
  6. Forward Integral Algorithm
  7. Numerical Corrector Algorithm
  8. Extension to Convolutional Neural Networks and Residual Neural Networks
  9. Discussion
  10. Appendix: Forward Integral Algorithm
  11. Appendix: Corrector Algorithm
  12. Appendix: Experiment

Key Result

Lemma 2.1

For a one-layer Neural Network $\psi$ defined on a closed interval $[a, b]$, the integral of $\psi$ can be expressed as: where $z = (z_{1}, \dots, z_{n_{1}})^\intercal \in \mathbb{R}^{n_{1}}$ and $z_{i} = \int_{a}^{x} \sigma(w_{i}t + b^{(1)}_{i}) dt.$

Figures (2)

  • Figure 1: Numerical Experiment
  • Figure 2: Integration Process

Theorems & Definitions (2)

  • Lemma 2.1
  • Theorem 2.2: arora2018understanding