Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Tree-SNE Tree Exists

Jack Kendrick

TL;DR

This work analyzes scale sensitivity in t-SNE and introduces Tree-SNE, a hierarchical embedding that reveals subclusters by progressively weakening the kernel tail via a one-parameter family $K_\alpha(d)$, yielding a tree-like embedding. By formulating the embedding as the zero-set of a gradient $F$ and proving that generically $F$ has constant rank, the authors apply the constant-rank level-set theorem to show the zero-set is a smooth manifold of dimension $1+d(d+1)/2$ with a non-constant scaling parameter $\alpha$, guaranteeing a continuous Tree-SNE tree. They demonstrate practical utility on large high-dimensional datasets from the HathiTrust library and on a Shakespeare authorship dataset, illustrating macro- and micro-level clustering along tree branches. The results provide a principled, scale-aware framework for clustering with attraction–repulsion embeddings and open avenues for extending the approach to other methods with similar dynamics.

Abstract

The clustering and visualisation of high-dimensional data is a ubiquitous task in modern data science. Popular techniques include nonlinear dimensionality reduction methods like t-SNE or UMAP. These methods face the `scale-problem' of clustering: when dealing with the MNIST dataset, do we want to distinguish different digits or do we want to distinguish different ways of writing the digits? The answer is task dependent and depends on scale. We revisit an idea of Robinson & Pierce-Hoffman that exploits an underlying scaling symmetry in t-SNE to replace 2-dimensional with (2+1)-dimensional embeddings where the additional parameter accounts for scale. This gives rise to the t-SNE tree (short: tree-SNE). We prove that the optimal embedding depends continuously on the scaling parameter for all initial conditions outside a set of measure 0: the tree-SNE tree exists. This idea conceivably extends to other attraction-repulsion methods and is illustrated on several examples.

The Tree-SNE Tree Exists

TL;DR

This work analyzes scale sensitivity in t-SNE and introduces Tree-SNE, a hierarchical embedding that reveals subclusters by progressively weakening the kernel tail via a one-parameter family , yielding a tree-like embedding. By formulating the embedding as the zero-set of a gradient and proving that generically has constant rank, the authors apply the constant-rank level-set theorem to show the zero-set is a smooth manifold of dimension with a non-constant scaling parameter , guaranteeing a continuous Tree-SNE tree. They demonstrate practical utility on large high-dimensional datasets from the HathiTrust library and on a Shakespeare authorship dataset, illustrating macro- and micro-level clustering along tree branches. The results provide a principled, scale-aware framework for clustering with attraction–repulsion embeddings and open avenues for extending the approach to other methods with similar dynamics.

Abstract

The clustering and visualisation of high-dimensional data is a ubiquitous task in modern data science. Popular techniques include nonlinear dimensionality reduction methods like t-SNE or UMAP. These methods face the `scale-problem' of clustering: when dealing with the MNIST dataset, do we want to distinguish different digits or do we want to distinguish different ways of writing the digits? The answer is task dependent and depends on scale. We revisit an idea of Robinson & Pierce-Hoffman that exploits an underlying scaling symmetry in t-SNE to replace 2-dimensional with (2+1)-dimensional embeddings where the additional parameter accounts for scale. This gives rise to the t-SNE tree (short: tree-SNE). We prove that the optimal embedding depends continuously on the scaling parameter for all initial conditions outside a set of measure 0: the tree-SNE tree exists. This idea conceivably extends to other attraction-repulsion methods and is illustrated on several examples.
Paper Structure (15 sections, 7 theorems, 14 equations, 6 figures)

This paper contains 15 sections, 7 theorems, 14 equations, 6 figures.

Table of Contents

  1. Introduction and Results
  2. The problem of clustering.
  3. The problem of scale
  4. Tree-SNE
  5. Main Result
  6. Preliminaries
  7. T-SNE
  8. Tree-SNE
  9. Proof of Continuous Structure
  10. A Discussion of Assumption \ref{['ass:rank']}
  11. Proof of Theorem.
  12. Examples
  13. English and American Literature
  14. The Anti-Stratfordian Theory
  15. Conclusion

Key Result

Theorem 1

Suppose $(y_1,\ldots, y_n)\in{\mathbb R}^{nd}$ is a t-SNE embedding in ${\mathbb R}^d$ with parameter $\alpha,$ Under mild conditions, for each $\alpha'$ in some neighbourhood of $\alpha$ there is a t-SNE embedding $(y_1', \ldots, y_n')\in{\mathbb R}^{nd}$ in a neighbourhood of $(y_1, \ldots, y_n).$

Figures (6)

  • Figure 1: A schematic of subclusters in the MNIST dataset.
  • Figure 2: Slices of the MNIST tree-SNE embedding are interpolated to create a tree. Each color corresponds to a digit.
  • Figure 3: Subbranches of the MNIST tree correspond to different handwriting styles. Images are the average of elements in the corresponding cluster.
  • Figure 4: Tree-SNE embedding of 100,000 works of American and English literature. The red points correspond to English literature and blue points correspond to American literature.
  • Figure 5: The trajectories of 109 works in the tree-SNE embedding in \ref{['fig:hathitrust-tree']}. These works form a cluster in the tree approximately halfway through the embedding and subsequently split into more informative subclusters towards the top of the tree.
  • ...and 1 more figures

Theorems & Definitions (11)

  • Theorem : The Tree-SNE Tree exists
  • Theorem : The Tree-SNE tree exists - refined
  • Theorem 3.1: Constant-Rank Level Set
  • Proposition : Rank Deficient Hessian
  • proof
  • Lemma 3.3
  • proof
  • Lemma 3.4
  • proof
  • Lemma 3.5
  • ...and 1 more