Spherical dimension

TL;DR

The paper presents the spherical dimension as a topological counterpart to the VC dimension by embedding the realizable-distribution space $\Delta(\mathcal{H})$ into antipodal structures and leveraging the Borsuk-Ulam framework. It proves foundational bounds such as $sd(\mathcal{H})\ge VC(\mathcal{H})-1$ and $sd(\mathcal{H})\ge VC^{*\mathrm{a}}(\mathcal{H})-2$, analyzes growth under products (e.g., $sd(\mathcal{U}^{m}_{n})\le mn$) and extremal embeddings, and develops a deformation-retraction machinery linking $\Delta_{\mathcal{E}}$ to cubical complexes for extremal classes. The framework yields four broad applications: disambiguations of margin-based linear classifiers, embeddings into extremal classes, stability and list replicability, and reductions to stochastic convex optimization, while connecting finite-sd questions to open problems on halfspaces with margin. Collectively, this work unifies diverse topological methods in learning theory, offering concrete bounds and structural insights with potential implications for compression, optimization reductions, and replicability guarantees.

Abstract

We introduce and study the spherical dimension, a natural topological relaxation of the VC dimension that unifies several results in learning theory where topology plays a key role in the proofs. The spherical dimension is defined by extending the set of realizable datasets (used to define the VC dimension) to the continuous space of realizable distributions. In this space, a shattered set of size d (in the VC sense) is completed into a continuous object, specifically a d-dimensional sphere of realizable distributions. The spherical dimension is then defined as the dimension of the largest sphere in this space. Thus, the spherical dimension is at least the VC dimension. The spherical dimension serves as a common foundation for leveraging the Borsuk-Ulam theorem and related topological tools. We demonstrate the utility of the spherical dimension in diverse applications, including disambiguations of partial concept classes, reductions from classification to stochastic convex optimization, stability and replicability, and sample compression schemes. Perhaps surprisingly, we show that the open question posed by Alon, Hanneke, Holzman, and Moran (FOCS 2021) of whether there exist non-trivial disambiguations for halfspaces with margin is equivalent to the basic open question of whether the VC and spherical dimensions are finite together.

Figures (21)

• Figure 1: Space of realizable distributions of a four-concept class shattering two points.
• Figure 2: Cubical complex (in blue) of a threshold on $3$ points $\mathcal{E} = \{---, --+, -++, +++\}$, embedded into its space of realizable distributions.
• Figure 3: Hasse diagram for spherical dimensions and related values. For any class $\mathcal{H}$, the respective values are non-decreasing with respect to this partial order. The wavy line indicates rough equivalence. The parameter $\mathtt{disamb}$ is defined in the proof of \ref{['th-sd-disamb']}; roughly, how big of a sphere can a class disambiguate.
• Figure 4: Simplicial complex $\Delta_\mathcal{H}$ for a class $\mathcal{H}$ on the domain $\{1,2,3\}$ with $4$ concepts: $---$, $-+-$, $+--$, and $++-$. The subcomplex $\Delta^\mathrm{ant}_\mathcal{H}$ is the boundary of $\Delta(\mathcal{H})$, drawn in thick blue lines.
• Figure 5: Barycentric subdivision of a $2$-dimensional simplex $\{1,2,3\}$ and a barycentric subdivision of a $1$-dimensional simplicial complex $\{12, 1^*2, 1^*2^*,12^*\}$ that extends the antipodality map $v \mapsto v^*$.

Theorems & Definitions (60)

• Definition 1: Spherical dimension
• Theorem 1: Rough equivalence between disambiguations and spherical dimension
• Lemma 2
• Lemma 3
• Lemma 4
• Conjecture 5
• Conjecture 6
• Conjecture 7
• Theorem 2
• Lemma 8: Assouad's bounds for antipodal shattering