Faster k-Medoids Clustering: Improving the PAM, CLARA, and CLARANS Algorithms
Erich Schubert, Peter J. Rousseeuw
TL;DR
This work addresses the slow SWAP phase of Partitioning Around Medoids (PAM) for arbitrary dissimilarities by introducing caching and reordering strategies (FastPAM1) that yield an empirical $O(k)$ speedup. It further accelerates convergence by allowing multiple swaps per iteration (FastPAM2) and by replacing the BUILD initialization with a linear-time LAB approach, together forming FastPAM. The improvements extend naturally to PAM-based variants like CLARA and CLARANS, delivering substantial speedups (often several orders of magnitude) while preserving solution quality. Overall, these methods enable PAM-family clustering on much larger datasets and higher cluster counts, with practical open-source implementations for ELKI and R.”
Abstract
Clustering non-Euclidean data is difficult, and one of the most used algorithms besides hierarchical clustering is the popular algorithm Partitioning Around Medoids (PAM), also simply referred to as k-medoids. In Euclidean geometry the mean-as used in k-means-is a good estimator for the cluster center, but this does not hold for arbitrary dissimilarities. PAM uses the medoid instead, the object with the smallest dissimilarity to all others in the cluster. This notion of centrality can be used with any (dis-)similarity, and thus is of high relevance to many domains such as biology that require the use of Jaccard, Gower, or more complex distances. A key issue with PAM is its high run time cost. We propose modifications to the PAM algorithm to achieve an O(k)-fold speedup in the second SWAP phase of the algorithm, but will still find the same results as the original PAM algorithm. If we slightly relax the choice of swaps performed (at comparable quality), we can further accelerate the algorithm by performing up to k swaps in each iteration. With the substantially faster SWAP, we can now also explore alternative strategies for choosing the initial medoids. We also show how the CLARA and CLARANS algorithms benefit from these modifications. It can easily be combined with earlier approaches to use PAM and CLARA on big data (some of which use PAM as a subroutine, hence can immediately benefit from these improvements), where the performance with high k becomes increasingly important. In experiments on real data with k=100, we observed a 200-fold speedup compared to the original PAM SWAP algorithm, making PAM applicable to larger data sets as long as we can afford to compute a distance matrix, and in particular to higher k (at k=2, the new SWAP was only 1.5 times faster, as the speedup is expected to increase with k).