Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Triangulating PL functions and the existence of efficient ReLU DNNs

Danny Calegari

TL;DR

The paper tackles the problem of efficiently representing piecewise-linear functions with compact support by triangulating their graphs in a way that yields a fixed, shallow ReLU neural network architecture. It develops a simplex-function decomposition tied to a degree-1 triangulation of the relative homology class bounded by the support and the graph, enabling a universal construction that computes all functions of bounded complexity. A key result is that any $f\in\mathcal{PL}(d,n)$ can be written as a sum of at most $2n$ simplex functions, with each simplex function expressible as a max-min of linear forms, realized by a fixed network of depth $\le 2\log_2(d)+2$; thus, $f$ can be computed by combining $2n$ copies of a small subnetwork. The paper further connects computational complexity to geometry by using hyperbolic geometry to bound the number of terms in the representation via volume considerations in $\mathbb{H}^{d+1}$, linking representation size to geometric invariants such as $V(f)$ and $v_{d+1}$. Overall, the work provides a constructive, topologically informed view on the efficiency of ReLU networks for PL functions and establishes explicit architectural bounds.

Abstract

We show that every piecewise linear function $f:R^d \to R$ with compact support a polyhedron $P$ has a representation as a sum of so-called `simplex functions'. Such representations arise from degree 1 triangulations of the relative homology class (in $R^{d+1}$) bounded by $P$ and the graph of $f$, and give a short elementary proof of the existence of efficient universal ReLU neural networks that simultaneously compute all such functions $f$ of bounded complexity.

Triangulating PL functions and the existence of efficient ReLU DNNs

TL;DR

The paper tackles the problem of efficiently representing piecewise-linear functions with compact support by triangulating their graphs in a way that yields a fixed, shallow ReLU neural network architecture. It develops a simplex-function decomposition tied to a degree-1 triangulation of the relative homology class bounded by the support and the graph, enabling a universal construction that computes all functions of bounded complexity. A key result is that any can be written as a sum of at most simplex functions, with each simplex function expressible as a max-min of linear forms, realized by a fixed network of depth ; thus, can be computed by combining copies of a small subnetwork. The paper further connects computational complexity to geometry by using hyperbolic geometry to bound the number of terms in the representation via volume considerations in , linking representation size to geometric invariants such as and . Overall, the work provides a constructive, topologically informed view on the efficiency of ReLU networks for PL functions and establishes explicit architectural bounds.

Abstract

We show that every piecewise linear function with compact support a polyhedron has a representation as a sum of so-called `simplex functions'. Such representations arise from degree 1 triangulations of the relative homology class (in ) bounded by and the graph of , and give a short elementary proof of the existence of efficient universal ReLU neural networks that simultaneously compute all such functions of bounded complexity.
Paper Structure (3 sections, 3 theorems, 1 equation)

This paper contains 3 sections, 3 theorems, 1 equation.

Table of Contents

  1. Introduction
  2. An approximation theorem
  3. Hyperbolic geometry

Key Result

Lemma 2.2

Let $\tau(\Delta):\mathbb R^d \to \mathbb R$ be a simplex function. Then there are at most $d^2$linear functions $g_1,\cdots,g_{d^2}:\mathbb R^d \to \mathbb R$ so that Consequently there is a fixed ReLU neural network $T$ with so that for every simplex function $\tau(\Delta)$ there is some assignment of weights and biases to $T$ that computes $\tau$.

Theorems & Definitions (9)

  • Definition 2.1: Simplex function
  • Lemma 2.2: Max-min normal form for simplex function
  • proof
  • Remark 2.3
  • Theorem 2.4
  • proof
  • Theorem 3.1
  • proof
  • Conjecture 3.2