Learning Self-Growth Maps for Fast and Accurate Imbalanced Streaming Data Clustering

TL;DR

The paper addresses imbalanced streaming data clustering (ISDC) where cluster sizes drift over time and the true number of clusters $k^*$ is unknown. It introduces SOHI, a two-stage framework that first learns a fast, incremental distribution representation via Self-Growth Maps (SGM) built from multiple triangular subnetworks, then employs Hierarchical Merging (HM) guided by a density-gap criterion to uncover imbalanced clusters and automatically select $\hat{k}^*$. HM uses 1D projections along cluster centers to evaluate density gaps and combines global compactness $\theta_k$ with global separability $\omega_k$ (via knee-point selection) to determine the optimal cluster count. Empirical results on 11 datasets, augmented by a Two-Layer Random Sampling (TLRS) streaming chunk generator, show that SOHI achieves superior clustering accuracy while maintaining competitive efficiency, outperforming fast baselines and remaining competitive with state-of-the-art static imbalanced clustering methods.

Abstract

Streaming data clustering is a popular research topic in data mining and machine learning. Since streaming data is usually analyzed in data chunks, it is more susceptible to encounter the dynamic cluster imbalance issue. That is, the imbalance ratio of clusters changes over time, which can easily lead to fluctuations in either the accuracy or the efficiency of streaming data clustering. Therefore, we propose an accurate and efficient streaming data clustering approach to adapt the drifting and imbalanced cluster distributions. We first design a Self-Growth Map (SGM) that can automatically arrange neurons on demand according to local distribution, and thus achieve fast and incremental adaptation to the streaming distributions. Since SGM allocates an excess number of density-sensitive neurons to describe the global distribution, it can avoid missing small clusters among imbalanced distributions. We also propose a fast hierarchical merging strategy to combine the neurons that break up the relatively large clusters. It exploits the maintained SGM to quickly retrieve the intra-cluster distribution pairs for merging, which circumvents the most laborious global searching. It turns out that the proposed SGM can incrementally adapt to the distributions of new chunks, and the Self-grOwth map-guided Hierarchical merging for Imbalanced data clustering (SOHI) approach can quickly explore a true number of imbalanced clusters. Extensive experiments demonstrate that SOHI can efficiently and accurately explore cluster distributions for streaming data.

Figures (6)

• Figure 1: Key problems in imbalanced streaming data clustering.
• Figure 2: Overview of the proposed SOHI framework.
• Figure 3: Effect of increasing data size on running time.
• Figure 4: ARI performance on 50 streaming chunks of LM dataset.
• Figure 5: NMI performance on 50 streaming chunks of LM dataset.