Robust Categorical Data Clustering Guided by Multi-Granular Competitive Learning

TL;DR

This work tackles clustering of purely categorical data, where the implicit distance space creates nested, multi-granular cluster structures that are hard to capture with traditional methods. It introduces MCDC, a two-stage framework combining MGCPL (multi-granular competitive penalization learning) to uncover a hierarchy of granular partitions and CAME (cluster aggregation based on MGCPL encoding) to encode these multi-granular affiliations into embeddings for final clustering. The approach yields automatic discovery of multiple valid clusterings, robust performance across diverse domains, and linear-time scalability suitable for large datasets and pre-partitioning in distributed settings. Empirical results on real and synthetic data demonstrate consistent improvements over state-of-the-art methods, with statistical significance and strong ablation support for the contribution of both MGCPL and CAME components.

Abstract

Data set composed of categorical features is very common in big data analysis tasks. Since categorical features are usually with a limited number of qualitative possible values, the nested granular cluster effect is prevalent in the implicit discrete distance space of categorical data. That is, data objects frequently overlap in space or subspace to form small compact clusters, and similar small clusters often form larger clusters. However, the distance space cannot be well-defined like the Euclidean distance due to the qualitative categorical data values, which brings great challenges to the cluster analysis of categorical data. In view of this, we design a Multi-Granular Competitive Penalization Learning (MGCPL) algorithm to allow potential clusters to interactively tune themselves and converge in stages with different numbers of naturally compact clusters. To leverage MGCPL, we also propose a Cluster Aggregation strategy based on MGCPL Encoding (CAME) to first encode the data objects according to the learned multi-granular distributions, and then perform final clustering on the embeddings. It turns out that the proposed MGCPL-guided Categorical Data Clustering (MCDC) approach is competent in automatically exploring the nested distribution of multi-granular clusters and highly robust to categorical data sets from various domains. Benefiting from its linear time complexity, MCDC is scalable to large-scale data sets and promising in pre-partitioning data sets or compute nodes for boosting distributed computing. Extensive experiments with statistical evidence demonstrate its superiority compared to state-of-the-art counterparts on various real public data sets.

Figures (6)

• Figure 1: Three categorical features (i.e., "GPU Type", "GPU Usage" and "Memory Usage") of a data set describing different compute nodes.
• Figure 2: Comparison of clusters of numerical and categorical data. Since categorical data objects overlap on six points in (b), spheres with different radii are used to indicate the occurrence frequency of overlapping objects. The natural distance structure of categorical data leads to the nested multi-granular cluster effect (e.g., the green cluster is composed of two clusters with different granularity), which brings difficulties to cluster analysis.
• Figure 3: Pipeline of the proposed method. MGCPL starts its learning with a relatively large initial $k_0$. The initialized clusters compete with each other to eliminate less important ones and obtain $k_1$. By inheriting $k_1$ as the initialization, the learning is re-lunched by clearing the parameters that guide the convergence. Such a process is recursively implemented until converges at $k_\sigma$ where the $k_\sigma$ prominent coarse-grained clusters cannot be further eliminated. The multi-granular results can be utilized for nested cluster distribution analysis, and can also be aggregated by CAME to accurately partition $X$ into $k$ clusters.
• Figure 4: Comparison of MCDC and its four ablated versions, i.e., MCDC$^4$, MCDC$^3$, MCDC$^2$, and MCDC$^1$, which are obtained by removing the weighting mechanism of CAME, the whole CAME, multi-granular learning mechanism of MGCPL, and the whole MGCPL from MCDC in turn.
• Figure 5: Different numbers of clusters learned by MGCPL. Blue dots indicate the number of clusters when MGCPL temporarily converges under the current cluster granularity. Red stars indicate the true number of clusters $k^*$.