A Survey on Deep Clustering: From the Prior Perspective

TL;DR

The surveyed work reframes deep clustering through a priors lens, arguing that progress hinges on how prior knowledge—six identified categories—guides feature learning and cluster assignment in the absence of labels. It traces a trajectory from structure- and distribution-based priors to augmentation-invariance, neighborhood consistency, pseudo-labeling, and external knowledge, with a demonstrated performance ladder across five benchmark datasets. The authors provide a benchmark-driven evaluation, highlight state-of-the-art gains from successive priors, and discuss practical applications and future challenges, including fine-grained, non-parametric, fair, and multi-view clustering. This perspective offers a concise, applicability-focused view that could steer future deep clustering research, particularly toward external knowledge and cross-modal signals. The work also notes potential synergies with external pre-trained models and language-vision frameworks to further enhance clustering capabilities, with broad implications for unsupervised learning in complex real-world data.

Abstract

Facilitated by the powerful feature extraction ability of neural networks, deep clustering has achieved great success in analyzing high-dimensional and complex real-world data. The performance of deep clustering methods is affected by various factors such as network structures and learning objectives. However, as pointed out in this survey, the essence of deep clustering lies in the incorporation and utilization of prior knowledge, which is largely ignored by existing works. From pioneering deep clustering methods based on data structure assumptions to recent contrastive clustering methods based on data augmentation invariances, the development of deep clustering intrinsically corresponds to the evolution of prior knowledge. In this survey, we provide a comprehensive review of deep clustering methods by categorizing them into six types of prior knowledge. We find that in general the prior innovation follows two trends, namely, i) from mining to constructing, and ii) from internal to external. Besides, we provide a benchmark on five widely-used datasets and analyze the performance of methods with diverse priors. By providing a novel prior knowledge perspective, we hope this survey could provide some novel insights and inspire future research in the deep clustering community.

Figures (6)

• Figure 1: Six categories of prior knowledge for deep clustering. (a) Structure Prior: data structure could reflect the semantic relation between instances. (b) Distribution Prior: instances from different clusters follow distinct data distributions. (c) Augmentation Invariance: samples augmented by the same instance have similar features. (d) Neighborhood Consistency: neighboring samples have consistent cluster assignments. (e) Pseudo Label: cluster assignments with high confidence are likely to be correct. (f) External Knowledge: abundant knowledge favorable to clustering exists in open-world data and models.
• Figure 2: The framework of distribution prior based methods. In addition to the standard continuous latent variable $\mathbf{z}_n$, generative deep clustering methods further introduce a discrete variable $\mathbf{z}_c$ to capture the cluster information.
• Figure 3: The framework of augmentation invariance based methods. Diverse transformations are first applied to augment the input data $x$, after which the shared deep neural network is utilized to extract features. The augmented samples of the same instance are encouraged to have similar features and cluster assignments.
• Figure 4: The framework of neighborhood consistency-based methods. Such a paradigm encourages neighboring samples $z_{i}$ and $z_{p}$ in the latent space to have consistent features and cluster assignments, which improves the compactness of clusters.
• Figure 5: The framework of pseudo-labeling based methods. Given features in the latent space, clustering algorithms such as K-means are performed to get pseudo labels. The pseudo labels, usually filtered by confidence, are then used as supervision signals to guide clustering.