Clust-Splitter $-$ an Efficient Nonsmooth Optimization-Based Algorithm for Clustering Large Datasets

TL;DR

Clust-Splitter addresses the minimum sum-of-squares clustering (MSSC) problem on very large datasets using a nonsmooth optimization (NSO) framework. It introduces three NSO formulations—a starting-point auxiliary problem, a 2-clustering auxiliary problem, and the main $k$-clustering problem—solved in an incremental fashion by splitting the most dissimilar cluster and solving subproblems with the limited memory bundle method (LMBM). The method demonstrates competitive accuracy and efficiency against state-of-the-art large-scale clustering algorithms across 18 real-world datasets and additional validation with synthetic data, with a particular strength for small cluster counts and robust performance for larger scales. The results position Clust-Splitter as a practical, open-source tool for real-time or near-real-time clustering on massive datasets, complementing existing NSO-based approaches like LMBM-Clust. Overall, Clust-Splitter achieves high-quality clustering with favorable computation times, and its intuitive starting-point strategy enhances convergence to near-global optima in nonconvex MSSC landscapes.

Abstract

Clustering is a fundamental task in data mining and machine learning, particularly for analyzing large-scale data. In this paper, we introduce Clust-Splitter, an efficient algorithm based on nonsmooth optimization, designed to solve the minimum sum-of-squares clustering problem in very large datasets. The clustering task is approached through a sequence of three nonsmooth optimization problems: two auxiliary problems used to generate suitable starting points, followed by a main clustering formulation. To solve these problems effectively, the limited memory bundle method is combined with an incremental approach to develop the Clust-Splitter algorithm. We evaluate Clust-Splitter on real-world datasets characterized by both a large number of attributes and a large number of data points and compare its performance with several state-of-the-art large-scale clustering algorithms. Experimental results demonstrate the efficiency of the proposed method for clustering very large datasets, as well as the high quality of its solutions, which are on par with those of the best existing methods.

Figures (23)

• Figure 1: The Clust-Splitter method. Note that if $k=2$, after solving the 2-clustering auxiliary problem, we set $\hat{{\boldsymbol x}} = (\hat{{\boldsymbol y}}^1,\hat{{\boldsymbol y}}^2)$ and skip the step with the $k$-clustering problem.
• Figure 2: Cases categorized by relative errors for small $k$-clustering problems, where $k \leq 5$. The total number of cases per algorithm is 72, comprising 18 datasets and four different numbers of clusters $(k=2, 3, 4, 5)$.
• Figure 3: Cases categorized by CPU times for small $k$-clustering problems, where $k \leq 5$. The total number of cases per algorithm is 72, comprising 18 datasets and four different numbers of clusters $(k=2, 3, 4, 5)$.
• Figure 4: Cases categorized by relative errors for large $k$-clustering problems, where $k \geq 10$. The total number of cases per algorithm is 72, comprising 18 datasets and four different numbers of clusters $(k=10, 15, 20, 25)$. Note that if an algorithm fails to find a solution within the 20-hour time limit, the case is assigned to the category 'Fail'.
• Figure 5: Cases categorized by CPU times for large $k$-clustering problems, where $k \geq 10$. The total number of cases per algorithm is 72, comprising 18 datasets and four different numbers of clusters $(k=10, 15, 20, 25)$. Note that the category 'CPU $>$ 20 h' denotes a failure to find a solution within the 20-hour time limit.

Theorems & Definitions (4)

• Remark 4.1
• Remark 4.2
• Theorem 4.1
• Theorem 4.2