Accelerating hyperbolic t-SNE

TL;DR

The paper tackles the prohibitive computational cost of hyperbolic t-SNE by introducing a polar quadtree–based acceleration tailored to the Poincaré disk. It develops a modified data structure and a fast gradient-approximation scheme that reduces per-iteration complexity toward $\mathcal{O}(n\log n)$ while preserving embedding quality, demonstrated across multiple datasets including large-scale ones. The approach is compatible with existing hyperbolic t-SNE variants (CO-SNE, h-SNE, Poincaré maps) and serves as a versatile building block for future hyperbolic embeddings. Practically, this enables fast, scalable hyperbolic visualization and analysis of hierarchical data on standard CPU hardware with minimal loss in local neighborhood preservation.

Abstract

The need to understand the structure of hierarchical or high-dimensional data is present in a variety of fields. Hyperbolic spaces have proven to be an important tool for embedding computations and analysis tasks as their non-linear nature lends itself well to tree or graph data. Subsequently, they have also been used in the visualization of high-dimensional data, where they exhibit increased embedding performance. However, none of the existing dimensionality reduction methods for embedding into hyperbolic spaces scale well with the size of the input data. That is because the embeddings are computed via iterative optimization schemes and the computation cost of every iteration is quadratic in the size of the input. Furthermore, due to the non-linear nature of hyperbolic spaces, Euclidean acceleration structures cannot directly be translated to the hyperbolic setting. This paper introduces the first acceleration structure for hyperbolic embeddings, building upon a polar quadtree. We compare our approach with existing methods and demonstrate that it computes embeddings of similar quality in significantly less time. Implementation and scripts for the experiments can be found at https://graphics.tudelft.nl/accelerating-hyperbolic-tsne.

Figures (12)

• Figure 1: The Barnes-Hut data structure, showing the quadtree and the hierarchy. The influence of the points $\mathbf{y}_1,\mathbf{y}_2,\mathbf{y}_3$ on point $\mathbf{y}_9$ is approximated using their midpoint $\mathbf{y}_{cell}$ and the cell diagonal $r_{cell}$. Modified from vanDerMaaten2014accelerating.
• Figure 2: The Poincaré disk model of hyperbolic space with a blue straight line in the left that appears curved and has infinitely many parallels to it. On the right a regular tiling illustrating how tiles visually shrink towards the edge of the disk, while keeping their area within hyperbolic space.
• Figure 3: Building a polar quadtree: Poincaré disk with seven points, initially split into four pie slices, and splitting one of them along the angular and radial direction.
• Figure 4: Left: Initial annulus root cell of a polar quadtree. Center: Splitting the polar quadrilaterals such that they represent equal hyperbolic area (top) or at the center of the embedding coordinates (bottom). Right: Two polar quads that have different elements as their respective longest internal distances, highlighted in blue.
• Figure 5: Effect of the two splitting choices on the polar quadtree, note the long pieces towards the center when splitting according to \ref{['equ:EqualAreaSplitting']} on the left, while cells are more compact when splitting according to \ref{['equ:EqualEmbeddingLengthsSplitting']} on the right.