Guaranteed Recovery of Unambiguous Clusters

TL;DR

This paper addresses the inherent ambiguity in clustering when the number of clusters is known but cluster structure is non-convex and density-variant. It introduces an information-theoretic framework with weak separability and LM-separability to characterize unambiguous $K$-clusterings and proposes Minimal Seed Expansion (MSE), a seed-based density-clustering algorithm that provably recovers the true clustering whenever these conditions hold. The algorithm operates by identifying $K$ high-density seeds and greedily expanding them to form full clusters, with a version that handles overlapping clusters; it runs in $O(|X|^3 \log|X|)$ time. Empirically, MSE demonstrates strong performance relative to established non-convex clustering methods across benchmarks and artificial datasets, with competitive ARI/NMI, robust handling of overlaps, and minimal parameter tuning, supported by an open-source implementation.

Abstract

Clustering is often a challenging problem because of the inherent ambiguity in what the "correct" clustering should be. Even when the number of clusters $K$ is known, this ambiguity often still exists, particularly when there is variation in density among different clusters, and clusters have multiple relatively separated regions of high density. In this paper we propose an information-theoretic characterization of when a $K$-clustering is ambiguous, and design an algorithm that recovers the clustering whenever it is unambiguous. This characterization formalizes the situation when two high density regions within a cluster are separable enough that they look more like two distinct clusters than two truly distinct clusters in the $K$-clustering. The algorithm first identifies $K$ partial clusters (or "seeds") using a density-based approach, and then adds unclustered points to the initial $K$ partial clusters in a greedy manner to form a complete clustering. We implement and test a version of the algorithm that is modified to effectively handle overlapping clusters, and observe that it requires little parameter selection and displays improved performance on many datasets compared to widely used algorithms for non-convex cluster recovery.

Figures (5)

• Figure 1: A plot showing the sparsity value $\epsilon_3(x)$ of each point $x \in X$ in a 14 point one-dimensional clustering $C = [c_1, c_2]$ that is weakly separable and LM-separable for $N_p = 3$, implying that recovery is guaranteed. The red points correspond to points in $X_c^*$ for $c \in C$. The pink point corresponds to the local maximum $x \in X$ such that $\max_{y \in c(x): \; \epsilon_{N_p}(y) \leq \epsilon_{N_p}(x)} A(x, y) = A^\ell(C)$ (i.e. the local maximum that looks the most separated from its cluster). $C$ is weakly separable because $\epsilon^*(c_1) = 0.9$ and $\epsilon^*(c_2) = 0.5$, while $\min_{c, c' \in C} \epsilon(c, c') = 1.1$. $C$ is LM-separable because $A^\ell(C) = 2.0$ while $\min_{c, c' \in C} \min_{z \in X_c^*} A(z, c') = 4.4$.
• Figure 2: A plot showing the sparsity value $\epsilon_3(x)$ of each point $x \in X$ in a 60 point one-dimensional clustering $C = [c_1, c_2]$ that is weakly separable and LM-separable for $N_p = 3$, implying that recovery is guaranteed. The orange, red, and pink points correspond to local maxima. The red points correspond to points in $X_c^*$ for $c \in C$. The pink point corresponds to the local maximum $x \in X$ such that $\max_{y \in c(x): \; \epsilon_{N_p}(y) \leq \epsilon_{N_p}(x)} A(x, y) = A^\ell(C)$ (i.e. the local maximum that looks the most separated from its cluster). $C$ is weakly separable because $\epsilon^*(c_1) = 0.74$ and $\epsilon^*(c_2) = 1.14$, while $\min_{c, c' \in C} \epsilon(c, c') = 1.18$. $C$ is LM-separable because $A^\ell(C) = 20.49$ while $\min_{c, c' \in C} \min_{z \in X_c^*} A(z, c') = 44.27$.
• Figure 3: A clustering with 500 points that is weakly separable and LM-separable, but not strongly separable for $N_p = 5$, implying that recovery by our algorithm ("MSE") is guaranteed. We compare against spectral clustering (scikit-learn) with the $4$-nearest neighbors affinity matrix because for $N_p = 5$, our algorithm uses the $4$ nearest neighbors of a point to calculate density.
• Figure 4: Artificial datasets used in Table \ref{['table:simulation-results-sweep']} .
• Figure 5: t-SNE plots for the Digits dataset.

Theorems & Definitions (18)

• Definition 1
• Lemma 1
• Corollary 1
• Definition 2
• Theorem 1
• Lemma 2
• Definition 3
• Lemma 3
• Lemma 4
• Lemma 5