Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Local and global topological complexity measures OF ReLU neural network functions

J. Elisenda Grigsby, Kathryn Lindsey, Marissa Masden

TL;DR

This work develops a topological framework for ReLU neural networks by viewing realized functions as finite PL maps and applying a generalized PL Morse theory. It introduces a canonical polytopal model $\mathcal{K}(F)$ together with a strong deformation retract from the natural domain complex $|\mathcal{C}(F)|$ to a compact model, enabling finite computation of algebro-topological invariants like $H$-complexity. The authors prove that sublevel sets are homotopy equivalent across transversal thresholds via level-preserving deformation retracts, and they extend these results to both pointed and unpointed polyhedral complexes through essentialization. Finally, they show that local $H$-complexity can be arbitrarily large, constructing explicit network architectures that realize high local complexity, which highlights the rich topological structure of ReLU networks beyond generic PL Morse behavior.

Abstract

We apply a generalized piecewise-linear (PL) version of Morse theory due to Grunert-Kuhnel-Rote to define and study new local and global notions of topological complexity for fully-connected feedforward ReLU neural network functions, F: R^n -> R. Along the way, we show how to construct, for each such F, a canonical polytopal complex K(F) and a deformation retract of the domain onto K(F), yielding a convenient compact model for performing calculations. We also give a construction showing that local complexity can be arbitrarily high.

Local and global topological complexity measures OF ReLU neural network functions

TL;DR

This work develops a topological framework for ReLU neural networks by viewing realized functions as finite PL maps and applying a generalized PL Morse theory. It introduces a canonical polytopal model together with a strong deformation retract from the natural domain complex to a compact model, enabling finite computation of algebro-topological invariants like -complexity. The authors prove that sublevel sets are homotopy equivalent across transversal thresholds via level-preserving deformation retracts, and they extend these results to both pointed and unpointed polyhedral complexes through essentialization. Finally, they show that local -complexity can be arbitrarily large, constructing explicit network architectures that realize high local complexity, which highlights the rich topological structure of ReLU networks beyond generic PL Morse behavior.

Abstract

We apply a generalized piecewise-linear (PL) version of Morse theory due to Grunert-Kuhnel-Rote to define and study new local and global notions of topological complexity for fully-connected feedforward ReLU neural network functions, F: R^n -> R. Along the way, we show how to construct, for each such F, a canonical polytopal complex K(F) and a deformation retract of the domain onto K(F), yielding a convenient compact model for performing calculations. We also give a construction showing that local complexity can be arbitrarily high.
Paper Structure (19 sections, 49 theorems, 94 equations, 8 figures)

This paper contains 19 sections, 49 theorems, 94 equations, 8 figures.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Polyhedral complexes
  4. Feedforward ReLU neural networks: genericity, transversality and canonical polyhedral complex
  5. PL Morse theory and flat versus critical cells
  6. PL $n$--manifolds and PL Morse functions
  7. Flat cells, nontransversal thresholds, and general maps
  8. Flat cells for ReLU layer maps and proof of Theorem \ref{['thm:NotPLMorse']}
  9. Global and Local H-complexity
  10. Sublevel sets of finite PL maps are homotopy equivalent within transversal intervals
  11. The canonical polytopal complex of a pointed polyhedral complex
  12. Deformation retraction of a pointed polyhedral complex of $\mathbb{R}^n$ to its canonical polytopal complex
  13. The base map
  14. The canonical polytopal complex of unpointed polyhedral complexes
  15. Unpointed polyhedral complexes and their essentializations
  16. ...and 4 more sections

Key Result

Theorem 1

Let $F: \mathbb{R}^n \rightarrow \mathbb{R}$ be a finite PL function, and let $a \leq b \in \mathbb{R}$ be thresholds. If the polyhedral complex $\mathcal{C}(F)_{F_{\in [a,b]}}$ contains no flat cells, then the sublevel sets $F_{\leq a}$ and $F_{\leq b}$ are homotopy equivalent, as are the superleve

Figures (8)

  • Figure 1: Algebro-topological invariants of the sublevel sets $F_{\leq t}$ of a function $F: \mathbb{R}^n \rightarrow \mathbb{R}$ for varying $t \in \mathbb{R}$ give strong information about the complexity of the function.
  • Figure 2: Left: A PL non-degenerate critical point of index 1 of a function $\mathbb{R}^2 \to \mathbb{R}$. Right: A PL strongly regular point.
  • Figure 3: An unbounded, pointed polyhedral set expressed as the Minkowski sum of its characteristic cone and the convex hull of its vertices.
  • Figure 4: The $\textrm{base}_P$ map, visualized as sending each point in $P$ to the closest point in $K_P$ intersected with the negative characteristic cone of $P$ based at $x$.
  • Figure 5: An illustration of the setup for the proof of Lemma \ref{['l:deltaPcontinuous']}.
  • ...and 3 more figures

Theorems & Definitions (124)

  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Theorem 4
  • Lemma 2.1
  • Lemma 2.2
  • proof
  • Definition 3.1
  • Definition 3.2
  • Definition 3.3
  • ...and 114 more