CantorNet: A Sandbox for Testing Geometrical and Topological Complexity Measures

TL;DR

CantorNet introduces a fractal-inspired sandbox for testing geometrical and topological complexity in ReLU networks by constructing self-similar decision boundaries through a recursive generating function $A(x)=\\max\\{-3x+1,0,3x-2\\}$, yielding $R_k$ with tunable complexity. It establishes two equivalent representations: a recursion-based network with $O(k)$ neurons and a min-max (DNF-like) polyhedral decomposition with $O(2^{k})$ components, linked via a triadic expansion isomorphism that maps fractal digits to activation patterns. The paper argues that the recursion-based construction achieves minimal description length with Kolmogorov complexity $O(k)$, providing a rigorous framework to study activation-space geometry and its implications for data augmentation and adversarial robustness. Overall, CantorNet offers a principled, analyzable setting to probe how fractal-like structures in decision boundaries affect neural representations and their complexity metrics.

Abstract

Many natural phenomena are characterized by self-similarity, for example the symmetry of human faces, or a repetitive motif of a song. Studying of such symmetries will allow us to gain deeper insights into the underlying mechanisms of complex systems. Recognizing the importance of understanding these patterns, we propose a geometrically inspired framework to study such phenomena in artificial neural networks. To this end, we introduce \emph{CantorNet}, inspired by the triadic construction of the Cantor set, which was introduced by Georg Cantor in the $19^\text{th}$ century. In mathematics, the Cantor set is a set of points lying on a single line that is self-similar and has a counter intuitive property of being an uncountably infinite null set. Similarly, we introduce CantorNet as a sandbox for studying self-similarity by means of novel topological and geometrical complexity measures. CantorNet constitutes a family of ReLU neural networks that spans the whole spectrum of possible Kolmogorov complexities, including the two opposite descriptions (linear and exponential as measured by the description length). CantorNet's decision boundaries can be arbitrarily ragged, yet are analytically known. Besides serving as a testing ground for complexity measures, our work may serve to illustrate potential pitfalls in geometry-ignorant data augmentation techniques and adversarial attacks.

Figures (4)

• Figure 1: Left: The first iteration of the 1-1 correspondence between the ReLU net $\widetilde{\mathcal{N}}_A$, induced by the generating function $A$, and the triadic number expansion shows the intervals $I_1, I_2, I_3$ correspond to the digits $\{0, 1, 2\}$, respectively. Right: CantorNet is inspired by the construction of the Cantor set cantor1879ueber.
• Figure 2: Activation patterns $\pi_i$ induced by Eq. \ref{['eq:A']}. We skip neurons with unchanged values.
• Figure 3: The greyed regions can be represented as a 0-preimage of $\pi\circ h_1$ (left), and the union of the 0-preimages of $\pi\circ h_1$ and $\pi\circ h_2$ (right).
• Figure 4: Decision surfaces ($k=2,3$) of \ref{['eq:Rk']} with labeled functions $h_i$.

Theorems & Definitions (12)

• Definition 1
• Definition 2
• Lemma 1: Computational Complexity of Activation Patterns of $\mathcal{N}^{(k)}_A$
• proof
• Proposition 1
• Lemma 2
• proof
• Theorem 1
• Lemma 3
• proof