Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ReLU neural network approximation to piecewise constant functions

Zhiqiang Cai, Junpyo Choi, Min Liu

TL;DR

It is shown that a three-layer ReLU NN is sufficient to accurately approximate any piecewise constant function and establish its error bound.

Abstract

This paper studies the approximation property of ReLU neural networks (NNs) to piecewise constant functions with unknown interfaces in bounded regions in $\mathbb{R}^d$. Under the assumption that the discontinuity interface $Γ$ may be approximated by a connected series of hyperplanes with a prescribed accuracy $\varepsilon >0$, we show that a three-layer ReLU NN is sufficient to accurately approximate any piecewise constant function and establish its error bound. Moreover, if the discontinuity interface is convex, an analytical formula of the ReLU NN approximation with exact weights and biases is provided.

ReLU neural network approximation to piecewise constant functions

TL;DR

It is shown that a three-layer ReLU NN is sufficient to accurately approximate any piecewise constant function and establish its error bound.

Abstract

This paper studies the approximation property of ReLU neural networks (NNs) to piecewise constant functions with unknown interfaces in bounded regions in . Under the assumption that the discontinuity interface may be approximated by a connected series of hyperplanes with a prescribed accuracy , we show that a three-layer ReLU NN is sufficient to accurately approximate any piecewise constant function and establish its error bound. Moreover, if the discontinuity interface is convex, an analytical formula of the ReLU NN approximation with exact weights and biases is provided.

Paper Structure

This paper contains 13 sections, 3 theorems, 60 equations, 8 figures.

Table of Contents

  1. Introduction
  2. Three-layer ReLU neural network functions
  3. Main results
  4. Proof of Lemma \ref{['general lemma']}
  5. Convex $\hat{\Omega}_1$
  6. Non-Convex $\hat{\Omega}_1$
  7. Convex hull
  8. Convex decomposition
  9. Examples
  10. A two-dimensional circular interface
  11. A three-dimensional spherical interface
  12. A $10000$-dimensional hypercube interface
  13. A two-dimensional non-convex example

Key Result

Lemma 3.1

\newlabelgeneral lemma0 Let $\hat{\Gamma}=\partial\hat{\Omega}_1\cap \partial\hat{\Omega}_2$. There exists a $d$--$n_1$--$n_2$--1 ReLU NN function $\mathcal{N}$ such that where $n_1$ and $n_2$ are integers depending on, respectively, the number of the hyperplanes and convexity of $\hat{\Gamma}$, and $C(|\hat{\Gamma}|)$ is a positive constant depending on the $(d-1)$-dimensional measure of the in

Figures (8)

  • Figure 1: An approximation of the interface $\Gamma$
  • Figure 1: The subdomain $\hat{\Omega}_1$ is convex.
  • Figure 1: A convex example to illustrate Theorem \ref{['general theorem']} for the case $d=2$
  • Figure 2: The subdomain $\hat{\Omega}_1$ is non-convex.
  • Figure 2: A convex example to illustrate Theorem \ref{['general theorem']} for the case $d=3$
  • ...and 3 more figures

Theorems & Definitions (10)

  • Lemma 3.1
  • Proof 1
  • Theorem 3.2
  • Proof 2
  • Lemma 4.1
  • Proof 3
  • Proof 4: Proof of Lemma \ref{['general lemma']} for convex $\hat{\Omega}_1$
  • Proof 5: Proof of Lemma \ref{['general lemma']} for non-convex $\hat{\Omega}_1$
  • Proof 6: Proof of Lemma \ref{['general lemma']} for non-convex $\hat{\Omega}_1$
  • Remark 4.2