Hyperbolic Continuous Structural Entropy for Hierarchical Clustering

TL;DR

The paper tackles the lack of a global objective and reliance on fixed graphs in hierarchical clustering by introducing HypCSE, a differentiable framework that optimizes a continuous structural entropy objective in hyperbolic space. It defines CSE via hyperbolic lowest common ancestors, encodes graphs with hyperbolic graph neural networks, and decodes hierarchical structures from embeddings. A graph structure learning module with contrastive learning adaptively refines the graph during training, yielding superior hierarchical clustering performance across seven datasets. The work highlights the effectiveness of hyperbolic geometry for representing hierarchies and offers a scalable, end-to-end method for structure-enhanced clustering.

Abstract

Hierarchical clustering is a fundamental machine-learning technique for grouping data points into dendrograms. However, existing hierarchical clustering methods encounter two primary challenges: 1) Most methods specify dendrograms without a global objective. 2) Graph-based methods often neglect the significance of graph structure, optimizing objectives on complete or static predefined graphs. In this work, we propose Hyperbolic Continuous Structural Entropy neural networks, namely HypCSE, for structure-enhanced continuous hierarchical clustering. Our key idea is to map data points in the hyperbolic space and minimize the relaxed continuous structural entropy (SE) on structure-enhanced graphs. Specifically, we encode graph vertices in hyperbolic space using hyperbolic graph neural networks and minimize approximate SE defined on graph embeddings. To make the SE objective differentiable for optimization, we reformulate it into a function using the lowest common ancestor (LCA) on trees and then relax it into continuous SE (CSE) by the analogy of hyperbolic graph embeddings and partitioning trees. To ensure a graph structure that effectively captures the hierarchy of data points for CSE calculation, we employ a graph structure learning (GSL) strategy that updates the graph structure during training. Extensive experiments on seven datasets demonstrate the superior performance of HypCSE.

Figures (6)

• Figure 1: HypCSE overview. Graphs are encoded as hyperbolic embeddings by minimizing CSE. Partitioning trees are decoded from embeddings for hierarchical clustering.
• Figure 2: Framework of HypCSE. (I) In the hyperbolic hierarchical clustering module, we construct an anchor graph $G_a$ from the input data, encode it using $f(\cdot)$, and decode it into a binary partitioning tree for hierarchical clustering. (II) In the GSL module, we learn a leaner graph $G_l$ using graph learner $g(\cdot)$, update $G_a$ from $G_l$, and guide $g(\cdot)$ via contrastive learning.
• Figure 3: Parameter sensitivity on decay rate $\tau$ (DP $\%$).
• Figure 4: Parameter sensitivity on $\eta_1$ and $k$ (DP $\%$).
• Figure 5: Zoo

Theorems & Definitions (7)

• Definition 1: Structural Entropy (li2016structural)
• Lemma 2: Minimum Structural Entropy (zhang2021supertad)
• Lemma 3: Connection to Graph-Based Clustering-Appendix A.1
• Definition 4: Structural Entropy via LCA
• Theorem 5: Equivalence (pan2021information)
• Lemma 6: Descendant via LCA-Appendix A.2
• Lemma 7: Distance of geodesic to origin chami2020trees