Metric $k$-clustering using only Weak Comparison Oracles

TL;DR

This framework demonstrates how noisy, low-cost oracles, such as those derived from large language models, can be systematically integrated into scalable clustering algorithms.

Abstract

Clustering is a fundamental primitive in unsupervised learning. However, classical algorithms for $k$-clustering (such as $k$-median and $k$-means) assume access to exact pairwise distances -- an unrealistic requirement in many modern applications. We study clustering in the \emph{Rank-model (R-model)}, where access to distances is entirely replaced by a \emph{quadruplet oracle} that provides only relative distance comparisons. In practice, such an oracle can represent learned models or human feedback, and is expected to be noisy and entail an access cost. Given a metric space with $n$ input items, we design randomized algorithms that, using only a noisy quadruplet oracle, compute a set of $O(k \cdot \mathsf{polylog}(n))$ centers along with a mapping from the input items to the centers such that the clustering cost of the mapping is at most constant times the optimum $k$-clustering cost. Our method achieves a query complexity of $O(n\cdot k \cdot \mathsf{polylog}(n))$ for arbitrary metric spaces and improves to $O((n+k^2) \cdot \mathsf{polylog}(n))$ when the underlying metric has bounded doubling dimension. When the metric has bounded doubling dimension we can further improve the approximation from constant to $1+\varepsilon$, for any arbitrarily small constant $\varepsilon\in(0,1)$, while preserving the same asymptotic query complexity. Our framework demonstrates how noisy, low-cost oracles, such as those derived from large language models, can be systematically integrated into scalable clustering algorithms.

Figures (8)

• Figure 1: Let $\mathsf{s} \in \mathcal{S}_i^{(1)}$ be the green vertex. The vertices in $\textsc{kernel}_i(\mathsf{s})$ are shown in red, and those in $\textsc{Guard}_i(\mathsf{s})$ are shown in blue. All remaining vertices in $\mathsf{V}$ are depicted as black points. The combined set $\textsc{kernel}_i(\mathsf{s}) \cup \textsc{Guard}_i(\mathsf{s})$ consists of vertices that are close to $\mathsf{s}$ in rank relative to all vertices in $\mathsf{V}$, with those in $\textsc{kernel}_i(\mathsf{s})$ being closer to $\mathsf{s}$ than those in $\textsc{Guard}_i(\mathsf{s})$. All black vertices inside the red circle will be filtered out. No vertex outside the blue circle will be filtered out. Some vertices between the red and blue circles may also be filtered out.
• Figure 2: Comparison of the $k$-means clustering results obtained by our method against the baseline and the optimal clustering.
• Figure 3: Adult
• Figure 4: Credit
• Figure 5: Clustering cost varying $k$

Theorems & Definitions (47)

• Lemma 2.1
• Lemma 2.2
• Theorem 3.1
• Theorem 3.2
• Theorem 3.3
• Theorem A.1
• proof
• Lemma B.1: Isolation
• proof
• Lemma B.2