Settling Time vs. Accuracy Tradeoffs for Clustering Big Data

TL;DR

The conditions in which coresets are necessary for preserving cluster validity as well as the settings in which faster, cruder sampling strategies are sufficient are shown, providing a comprehensive theoretical and practical blueprint for effective clustering regardless of data size.

Abstract

We study the theoretical and practical runtime limits of k-means and k-median clustering on large datasets. Since effectively all clustering methods are slower than the time it takes to read the dataset, the fastest approach is to quickly compress the data and perform the clustering on the compressed representation. Unfortunately, there is no universal best choice for compressing the number of points - while random sampling runs in sublinear time and coresets provide theoretical guarantees, the former does not enforce accuracy while the latter is too slow as the numbers of points and clusters grow. Indeed, it has been conjectured that any sensitivity-based coreset construction requires super-linear time in the dataset size. We examine this relationship by first showing that there does exist an algorithm that obtains coresets via sensitivity sampling in effectively linear time - within log-factors of the time it takes to read the data. Any approach that significantly improves on this must then resort to practical heuristics, leading us to consider the spectrum of sampling strategies across both real and artificial datasets in the static and streaming settings. Through this, we show the conditions in which coresets are necessary for preserving cluster validity as well as the settings in which faster, cruder sampling strategies are sufficient. As a result, we provide a comprehensive theoretical and practical blueprint for effective clustering regardless of data size. Our code is publicly available and has scripts to recreate the experiments.

Figures (5)

• Figure 1: Mean runtime over five runs as we vary $k$ for sensitivity sampling and Fast-Coresets. Bars are $k=50, 100, 200, 400$; y-axis is log-scale.
• Figure 2: Top: The effect of the $m$-scalar on coreset distortion for real-world datasets. This is a visualization of the data in Table \ref{['tbl:distortion']}. Bottom: The effect of the $m$-scalar on the algorithm runtime for real-world datasets. All values are the mean over 5 runs. The three bars represent samples of size $m=40k, 80k$.
• Figure 3: The results of lightweight and fast-coreset constructions on a 2D Gaussian mixture dataset of $n=100K$ points with clusters of varying size. The circled cluster has $\sim 400$ points and coresets have 200 points. Left: Original multivariate-Gaussian dataset. Middle: Lightweight coresets fail to capture the cluster of $\sim$ 400 points. Right: Sensitivity sampling with $j=k$ identifies all of the clusters.
• Figure 4: Sample coreset distortions under $k$-median for one run on each dataset. Bars within each dataset correspond to $m=40k, 60k, 80k$.
• Figure 5: Top: Coreset distortion on the $k$-means task in the streaming and non-streaming settings. This is a visualization of the data in Table \ref{['tbl:composition']}. Bottom: Coreset construction runtimes in the streaming and non-streaming settings for the linear and sub-linear complexity coreset algorithms. Bars are [Streaming, Non-Streaming].

Theorems & Definitions (17)

• Definition 2.1
• Lemma 2.2: Lemma 11.9 in har2011geometric
• Corollary 3.2
• Lemma 4.1
• Lemma 4.2
• Lemma 4.3
• proof
• Proposition 4.4
• proof
• Lemma 4.5