Combinatorial Regularity for Relatively Perfect Discrete Morse Gradient Vector Fields of ReLU Neural Networks

TL;DR

This work addresses the problem of extracting topological information from ReLU neural networks by leveraging PL Morse theory on the canonical polyhedral complex $\\mathcal{C}(F)$ and translating it into discrete Morse theory.It introduces a constructive scheme that yields a relatively perfect discrete gradient vector field on $\\mathcal{C}(F)$, preserving the sublevel set topology of the PL Morse function induced by the network.The main result, a canonical construction of a relatively perfect discrete gradient vector field (Theorem: relativelyperfectDGVF), provides an algorithmic pathway to obtain a discrete Morse function whose critical cells reflect PL Morse critical points.Additionally, the paper presents realizability results for shallow networks and develops computational strategies for local gradient computation and gradient-flow tracking to enable scalable topology-based analyses of neural decision regions.Overall, the work bridges PL Morse and discrete Morse methodologies in the context of ReLU networks, offering a framework with potential practical impact on the topological analysis of neural network decision boundaries.

Abstract

One common function class in machine learning is the class of ReLU neural networks. ReLU neural networks induce a piecewise linear decomposition of their input space called the canonical polyhedral complex. It has previously been established that it is decidable whether a ReLU neural network is piecewise linear Morse. In order to expand computational tools for analyzing the topological properties of ReLU neural networks, and to harness the strengths of discrete Morse theory, we introduce a schematic for translating between a given piecewise linear Morse function (e.g. parameters of a ReLU neural network) on a canonical polyhedral complex and a compatible (``relatively perfect") discrete Morse function on the same complex. Our approach is constructive, producing an algorithm that can be used to determine if a given vertex in a canonical polyhedral complex corresponds to a piecewise linear Morse critical point. Furthermore we provide an algorithm for constructing a consistent discrete Morse pairing on cells in the canonical polyhedral complex which contain this vertex. We additionally provide some new realizability results with respect to sublevel set topology in the case of shallow ReLU neural networks.

Figures (9)

• Figure 1: Upper left: The stars of the indicated vertices overlap. Upper right: One possible polyhedral construction of the local stars of the indicated vertices. Bottom left: The lower stars of the indicated vertices, given the indicated gradient directions. Observe the lower stars are necessarily disjoint. Bottom right: The local lower stars of the indicated vertices have simpler combinatorial type, but the cells are in bijection with the cells in the lower star.
• Figure 2: Upper left: The links of the indicated vertices overlap and have arbitrary combinatorial type. Upper right: The local links of the indicated vertices. Bottom: The local lower links of the indicated vertices, given the indicated $\nabla F$-orientations on edges.
• Figure 3: Left: A $\nabla F$-induced orientation on the edges of a polyhedral complex, with PL Morse critical points indicated. There are one index-zero critical point, two index-one critical points, and one index-two critical point. Right: One possible set of critical cells which would make a discrete gradient vector field on the polyhedral complex relatively perfect to $F$.
• Figure 4: (Left) A portion of a canonical polyhedral complex $\mathcal{C}(F)$ given as a bent hyperplane arrangement. The two "bent" hyperplanes which are not lines are given distinct colors. (Right) A plausible $\nabla F$ orientation on the edges of this $\mathcal{C}(F)$, and a plausible level set, marked in red.
• Figure 5: The union of the local star and local link of a vertex $v$ is a cross-polytope.

Theorems & Definitions (48)

• Definition 1: Cone, Cone Neighborhood, cf. RSgrunert
• Remark 1
• Definition 2: Star, Local Star, Lower Star, Local Lower Star
• Definition 3: Link, Local Link, Local Lower Link
• Definition 4: $\nabla F$ orientation, GrigsbyLindsey
• Lemma 1
• proof
• Remark 2
• Lemma 2: Cf. GLMas Lemma 6.1
• Corollary 1