Minimax Rates for Hyperbolic Hierarchical Learning
Divit Rawal, Sriram Vishwanath
TL;DR
On depth-$R$ trees with branching factor $m$, the paper shows an exponential separation in sample complexity between bounded-radius Euclidean embeddings and constant-distortion hyperbolic embeddings under Lipschitz regularization. The Euclidean obstruction arises from a volumetric collision which forces Lipschitz constants to grow as $\exp(\Omega(R/k))$, yielding exponential sample complexity, while hyperbolic embeddings admit $O(1)$-Lipschitz realizability and a matching $\Theta(mR\log m)$ rate. A geometry-agnostic low-rank bottleneck shows that any rank-$k$ predictor captures only $O(k)$ hierarchical contrasts, underscoring limits of compact representations. Together, the results establish a minimax-optimal, geometry-aware separation: hyperbolic representations are statistically efficient for hierarchical targets, whereas Euclidean representations incur exponential penalties unless dimension scales accordingly.
Abstract
We prove an exponential separation in sample complexity between Euclidean and hyperbolic representations for learning on hierarchical data under standard Lipschitz regularization. For depth-$R$ hierarchies with branching factor $m$, we first establish a geometric obstruction for Euclidean space: any bounded-radius embedding forces volumetric collapse, mapping exponentially many tree-distant points to nearby locations. This necessitates Lipschitz constants scaling as $\exp(Ω(R))$ to realize even simple hierarchical targets, yielding exponential sample complexity under capacity control. We then show this obstruction vanishes in hyperbolic space: constant-distortion hyperbolic embeddings admit $O(1)$-Lipschitz realizability, enabling learning with $n = O(mR \log m)$ samples. A matching $Ω(mR \log m)$ lower bound via Fano's inequality establishes that hyperbolic representations achieve the information-theoretic optimum. We also show a geometry-independent bottleneck: any rank-$k$ prediction space captures only $O(k)$ canonical hierarchical contrasts.