Fast and explainable clustering based on sorting

TL;DR

This paper tackles scalable clustering by proposing CLASSIX, a two-phase algorithm that first sorts data along the first principal component and greedily aggregates nearby points within a radius $R$, then merges the resulting groups into clusters using either a distance-based or a density-based criterion. The method achieves near-linear time and linear space in many practical settings and provides intuitive explanations for the resulting clusters, aided by its simple two-parameter control: the radius $R$ and the minimum cluster size $minPts$. Through extensive experiments on synthetic and real data, CLASSIX demonstrates competitive performance against state-of-the-art methods (e.g., k-means++, DBSCAN, HDBSCAN, Quickshift++) across varying shapes, dimensionalities, and sizes, while maintaining fast runtimes and robust clustering quality. The work also emphasizes explainability and offers outlier handling and out-of-sample extension capabilities, highlighting its practical applicability to large-scale or dynamic datasets and its potential for interpretable clustering results.

Abstract

We introduce a fast and explainable clustering method called CLASSIX. It consists of two phases, namely a greedy aggregation phase of the sorted data into groups of nearby data points, followed by the merging of groups into clusters. The algorithm is controlled by two scalar parameters, namely a distance parameter for the aggregation and another parameter controlling the minimal cluster size. Extensive experiments are conducted to give a comprehensive evaluation of the clustering performance on synthetic and real-world datasets, with various cluster shapes and low to high feature dimensionality. Our experiments demonstrate that CLASSIX competes with state-of-the-art clustering algorithms. The algorithm has linear space complexity and achieves near linear time complexity on a wide range of problems. Its inherent simplicity allows for the generation of intuitive explanations of the computed clusters.

Figures (12)

• Figure 1: Illustration of a point cloud aggregated into $\ell=6$ groups (dashed circles) and merged into $k=3$ clusters (labelled by colour). The data points are ordered along the direction of the first principal component indicated by the arrow, and the six starting points of the groups (which are also data points) follow that same order.
• Figure 2: Demonstration of the outlier treatment. We apply CLASSIX to synthetic data with $n=2500$ points using the parameters $R=0.1$ and $\text{minPts}=5$. Left: Without outlier reassignment, 27 clusters are identified, four of which are genuine ground-truth clusters (shown in colour) and each of the other 23 clusters have less than $\text{minPts}$ points (shown as the black dots). Right: The reassignment of the 23 outlier clusters results in four clusters of good quality.
• Figure 3: Illustration of CLASSIX clustering with out-of-sample data. Top: Ground truth labels of the train and test data (left and right, respectively). Bottom: CLASSIX clustering of the train and test data (left and right, respectively). The train vs test split is 90% vs 10%.
• Figure 4: Illustration of the model analyzed in section \ref{['sec:simpmod']}. Top: A Gaussian blob of $n=10^4$ points (red dots) in $d=2$ dimensions, with the horizontal components having a standard deviation of $1$ and the vertical components having a standard deviation of $s=0.3$. The radius for the aggregation is chosen as $R=0.5$. The blue dots are all points whose first component (first principal coordinate) is within a distance of $R$ from $c=0.6$, while the green points are within a distance of $R$ from the group center $c=[0.6,0]^T$ (shown as the black dot). Bottom left: The quotient $P_2/P_1$ corresponding to the ratio of green and blue points as a function of $c$ and $d$. A ratio closer to $1$ means that sorting-based aggregation is more efficient, with fewer points being revisited. Bottom right: $P_2/P_1$ as a function of $R$ and $d$.
• Figure 5: Performance evaluation of various clustering methods on the Gaussian blobs data. Top: We vary the number of data points from $n=5,000$ to $50,000$ and measure the adjusted Rand index (left) and timing (right) for each method. The feature dimension is fixed at $d=10$ in all cases. Bottom: Average number of distance calculations per data point in the CLASSIX aggregation phase with and without early termination.