Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Topology of Neural Network Superlevel Sets

Bahman Gharesifard

Abstract

We show that neural networks with activations satisfying a Riccati-type ordinary differential equation condition, an assumption arising in recent universal approximation results in the uniform topology, produce Pfaffian outputs on analytic domains with format controlled only by the architecture. Consequently, superlevel sets, as well as Lie bracket rank drop loci for neural network parameterized vector fields, admit architecture-only bounds on topological complexity, in particular on total Betti numbers, uniformly over all weights.

On the Topology of Neural Network Superlevel Sets

Abstract

We show that neural networks with activations satisfying a Riccati-type ordinary differential equation condition, an assumption arising in recent universal approximation results in the uniform topology, produce Pfaffian outputs on analytic domains with format controlled only by the architecture. Consequently, superlevel sets, as well as Lie bracket rank drop loci for neural network parameterized vector fields, admit architecture-only bounds on topological complexity, in particular on total Betti numbers, uniformly over all weights.
Paper Structure (7 sections, 7 theorems, 67 equations, 2 figures)

This paper contains 7 sections, 7 theorems, 67 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Neural Network Superlevel Sets: A Topological Problem
  3. Main Results
  4. Control Geometry
  5. Proof of main results
  6. Mathematical Preliminaries
  7. Main proofs

Key Result

Proposition 3.1

Suppose that $d=1$ and that $F:I\to\mathbb R$ be the output of a depth-$L$ neural network as in eq:nn-vector with widths $n_1,\dots,n_L$ and $\sigma\in\mathcal{A}_{\mathrm{quad},r}$, on an open interval $I$ where $F$ is analytic. Let Then either $F\equiv 0$ on $I$, or where $C_I>0$ is a constant depending on the interval $I$, but independent of the network weights and biases. In particular, the

Figures (2)

  • Figure 1: A scalar map $F:V\to\mathbb R$ and its decision region $D=\{F\ge 0\}=F^{-1}([0,\infty))$ are depicted schematically. The dashed curves represent the zero level sets.
  • Figure 2: Assuming analyticity, each change of sign can only occur at a zero of $F$, so the number of interval components is controlled by $\mathop{\mathrm{Zeros}}\nolimits(F;I)$.

Theorems & Definitions (18)

  • Definition 2.1
  • Definition 2.2
  • Proposition 3.1
  • Theorem 3.2
  • Theorem 3.3
  • Definition 4.1
  • Definition 4.2
  • Definition 4.3
  • Theorem 4.4
  • Proposition 4.5
  • ...and 8 more