Rethinking Divisive Hierarchical Clustering from a Distributional Perspective

TL;DR

Rethinking Divisive Hierarchical Clustering shows that objective-based DHC using set-oriented bisecting fails to preserve cluster integrity and ground-truth structure. It introduces H-$\mathcal{K}C$, a distributional-kernel driven method that treats clusters as distributions and optimizes a total similarity objective $TSC$, with a proven global lower-bound guarantee and linear-time complexity. The approach yields dendrograms that avoid unwarranted splitting, group similar clusters, and align with ground-truth regions, demonstrated on artificial data and Spatial Transcriptomics including HER2 and Slide-seq V2, where it outperforms baselines. This distribution-oriented framework offers scalable, shape-agnostic clustering that can reveal biologically meaningful hierarchical structure in complex datasets.

Abstract

We uncover that current objective-based Divisive Hierarchical Clustering (DHC) methods produce a dendrogram that does not have three desired properties i.e., no unwarranted splitting, group similar clusters into a same subset, ground-truth correspondence. This shortcoming has their root cause in using a set-oriented bisecting assessment criterion. We show that this shortcoming can be addressed by using a distributional kernel, instead of the set-oriented criterion; and the resultant clusters achieve a new distribution-oriented objective to maximize the total similarity of all clusters (TSC). Our theoretical analysis shows that the resultant dendrogram guarantees a lower bound of TSC. The empirical evaluation shows the effectiveness of our proposed method on artificial and Spatial Transcriptomics (bioinformatics) datasets. Our proposed method successfully creates a dendrogram that is consistent with the biological regions in a Spatial Transcriptomics dataset, whereas other contenders fail.

Figures (13)

• Figure 1: Dendrograms of Bisect-Kmeans, SpecWRSC and H-$\mathcal{K}C$ on the artificial dataset which contains clusters of arbitrary shapes and varied densities. $\wp$ is Dendrogram Purity heller2005bayesian (see the details in Appendix \ref{['appendix: purity']}).
• Figure 2: HER2 tumor dataset andersson2020spatial with ground-truth labels (a) and manual annotation plot of the tissue sample (e). Dendrograms of SpecWRSC (f), Bisect-Kmeans (g) and H-$\mathcal{K}C$ (h), and their flat clustering results shown in the leaf nodes are given in (b), (c) and (d), respectively. The solid black line indicates the result of the first split in (b)-(d). The Dendrogram Purity ($\wp$) is calculated in terms of the 'Invasive cancer' and 'Cancer in situ' regions. The flat clustering result of SpaGCN hu2021spagcn, a recent end-to-end deep learning flat clustering method which produces a comparable flat clustering of Bisect-Kmeans (c), is provided in Appendix \ref{['append: SpaGCN']}.
• Figure 3: The Slide-seq V2 dataset on mouse hippocampus stickels2021highly. Allen Brain Atlas (a) shows the three primary regional divisions: CTX, HPF and TH. The coronal mouse olfactory bulb from the Allen Brain Atlas in the central area is labelled as CA1sp and CA3sp. Dendrograms of Bisect-Kmeans (d) and H-$\mathcal{K}C$ (e), and their flat clustering results shown in the leaf nodes are given in (b) and (c), respectively. The solid black line indicates the result of the first split in (b) and the third split in (c).
• Figure 4: Critical difference diagram.
• Figure 5: Results of IDK vs GDK in H-$\mathcal{K}C$ on the artificial dataset. $\wp$ is Dendrogram Purity (DP).

Theorems & Definitions (10)

• Definition 5.1
• Definition 5.2
• Definition 5.3
• Lemma 6.1
• proof
• Theorem 6.2
• proof
• Proposition 6.3
• Proposition B.1
• proof