Clustering with Tangles: Algorithmic Framework and Theoretical Guarantees

TL;DR

The proposed algorithmic framework for clustering with tangles is hierarchical and induces the notion of a soft dendrogram, which can help explore the cluster structure of a dataset.

Abstract

Originally, tangles were invented as an abstract tool in mathematical graph theory to prove the famous graph minor theorem. In this paper, we showcase the practical potential of tangles in machine learning applications. Given a collection of cuts of any dataset, tangles aggregate these cuts to point in the direction of a dense structure. As a result, a cluster is softly characterized by a set of consistent pointers. This highly flexible approach can solve clustering problems in various setups, ranging from questionnaires over community detection in graphs to clustering points in metric spaces. The output of our proposed framework is hierarchical and induces the notion of a soft dendrogram, which can help explore the cluster structure of a dataset. The computational complexity of aggregating the cuts is linear in the number of data points. Thus the bottleneck of the tangle approach is to generate the cuts, for which simple and fast algorithms form a sufficient basis. In our paper we construct the algorithmic framework for clustering with tangles, prove theoretical guarantees in various settings, and provide extensive simulations and use cases. Python code is available on github.

Figures (21)

• Figure 1: We consider a set of points and six cuts. The left image visualizes one possible tangle (consistent orientation). The right image visualizes three additional tangles, that exist on (sub)sets of the same cuts. When constructing the tangle search tree, we would first obtain two tangles on the set $\{P_1, ..., P_4\}$: the purple and the green/blue tangle. The green and the blue tangle share the orientations of the more informative cuts but differ in bipartition $P_5$ and $P_6$, indicated by dashed arrows. Lower down in the hierarchy, we get three tangles on the whole set of cuts $\{P_1, ..., P_6\}$: green, blue and the black tangle visualized in the left picture.
• Figure 2: A soft dendrogram as possible post-processing of tangles (Appendix \ref{['sub:post_processing']}). The estimated probability that a point belongs to the respective cluster is given by $p$.
• Figure 3: Example of tangles for a dataset in $\mathbb{R}^2$.
• Figure 4: Tree $T^\ast$ with node and edge attributes for a fixed object $v_x\in V$.
• Figure 5: Frequencies of the hand-designed score $s_{npi}$ in the dataset.

Theorems & Definitions (23)

• Definition 1: Consistency and Tangles
• Theorem 2: Tangles recover the ground truth mindsets
• Theorem 3: Tangles recover the ground truth blocks
• Theorem 4: Non-identifiability
• Theorem 5: All cluster centers induce distinct tangles
• Theorem 6: All tangles point to distinct cluster centers
• Proposition 8: Assumption \ref{['ass:orientations']} satisfied with high probability
• proof
• Lemma 9: Mindsets give tangles
• proof