Adversarially robust clustering with optimality guarantees

TL;DR

This work tackles robust clustering for data drawn from sub-Gaussian mixtures in the presence of adversarial outliers. It introduces the k-medians-hybrid algorithm, which estimates centroids via the coordinatewise median while labeling via Euclidean distance, achieving optimal mislabeling guarantees in a constant number of iterations given mild initialization and adequate separation. The authors prove both mislabeling and centroid-estimation guarantees under an adversarial contamination model, showing that their method matches the best possible rates up to logarithmic factors and extends robustness to high-dimensional settings. Empirical results on synthetic and real datasets corroborate the theory, demonstrating strong robustness to outliers and competitive runtime compared to LP/SDP-based approaches.

Abstract

We consider the problem of clustering data points coming from sub-Gaussian mixtures. Existing methods that provably achieve the optimal mislabeling error, such as the Lloyd algorithm, are usually vulnerable to outliers. In contrast, clustering methods seemingly robust to adversarial perturbations are not known to satisfy the optimal statistical guarantees. We propose a simple robust algorithm based on the coordinatewise median that obtains the optimal mislabeling rate even when we allow adversarial outliers to be present. Our algorithm achieves the optimal error rate in constant iterations when a weak initialization condition is satisfied. In the absence of outliers, in fixed dimensions, our theoretical guarantees are similar to that of the Lloyd algorithm. Extensive experiments on various simulated and public datasets are conducted to support the theoretical guarantees of our method.

Figures (10)

• Figure 1: Comparison of clustering via $\ell_1$ and $\ell_2$ metrics after initializing at true centroids
• Figure 2: The effect of outlier proportions on all three algorithms.
• Figure 3: The effect of outlier variance on all three clustering algorithms.
• Figure 4: The effect of data dimension on all three clustering algorithms.
• Figure 5: The effect of outlier location on all three algorithms.

Theorems & Definitions (26)

• Remark 1
• Theorem 1
• Theorem 2
• Remark 2: Necessity of the assumptions on $\mathsf{SNR}$
• Remark 3: Robust initialization that meets our theoretical requirements
• Remark 4: Comments on the initialization conditions
• Remark 5: Convergence analysis of the $k$-medians-hybrid algorithm
• Remark 6: Runtime of our algorithm
• Theorem 3
• Remark 7