Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Introductory Survey to Autoencoder-based Deep Clustering -- Sandboxes for Combining Clustering with Deep Learning

Collin Leiber, Lukas Miklautz, Claudia Plant, Christian Böhm

TL;DR

This survey addresses the challenge of clustering high-dimensional data without labels by leveraging autoencoders to learn clustering-friendly, non-linear embeddings. It systematically categorizes AE-based deep clustering methods along two dimensions: optimization strategy (sequential, alternating, simultaneous, generative) and neural network type, emphasizing AE architectures as versatile building blocks. By detailing representative algorithms (e.g., AE+$k$-Means, DEN, AEC, DCN, ACe/DeC, DipEncoder, DEC, IDEC, DCEC, DKM) and their objective formulations, the paper clarifies how reconstruction and clustering losses are balanced to produce effective embeddings and cluster assignments. It also discusses practical considerations such as parameterization, data augmentation, and self-supervised losses, and highlights future research directions in hyperparameter selection, robustness, and interpretability. Overall, the work consolidates the AE-based DC landscape, providing a foundation for researchers to design clustering-focused deep models and apply them across diverse data modalities.

Abstract

Autoencoders offer a general way of learning low-dimensional, non-linear representations from data without labels. This is achieved without making any particular assumptions about the data type or other domain knowledge. The generality and domain agnosticism in combination with their simplicity make autoencoders a perfect sandbox for researching and developing novel (deep) clustering algorithms. Clustering methods group data based on similarity, a task that benefits from the lower-dimensional representation learned by an autoencoder, mitigating the curse of dimensionality. Specifically, the combination of deep learning with clustering, called Deep Clustering, enables to learn a representation tailored to specific clustering tasks, leading to high-quality results. This survey provides an introduction to fundamental autoencoder-based deep clustering algorithms that serve as building blocks for many modern approaches.

An Introductory Survey to Autoencoder-based Deep Clustering -- Sandboxes for Combining Clustering with Deep Learning

TL;DR

This survey addresses the challenge of clustering high-dimensional data without labels by leveraging autoencoders to learn clustering-friendly, non-linear embeddings. It systematically categorizes AE-based deep clustering methods along two dimensions: optimization strategy (sequential, alternating, simultaneous, generative) and neural network type, emphasizing AE architectures as versatile building blocks. By detailing representative algorithms (e.g., AE+-Means, DEN, AEC, DCN, ACe/DeC, DipEncoder, DEC, IDEC, DCEC, DKM) and their objective formulations, the paper clarifies how reconstruction and clustering losses are balanced to produce effective embeddings and cluster assignments. It also discusses practical considerations such as parameterization, data augmentation, and self-supervised losses, and highlights future research directions in hyperparameter selection, robustness, and interpretability. Overall, the work consolidates the AE-based DC landscape, providing a foundation for researchers to design clustering-focused deep models and apply them across diverse data modalities.

Abstract

Autoencoders offer a general way of learning low-dimensional, non-linear representations from data without labels. This is achieved without making any particular assumptions about the data type or other domain knowledge. The generality and domain agnosticism in combination with their simplicity make autoencoders a perfect sandbox for researching and developing novel (deep) clustering algorithms. Clustering methods group data based on similarity, a task that benefits from the lower-dimensional representation learned by an autoencoder, mitigating the curse of dimensionality. Specifically, the combination of deep learning with clustering, called Deep Clustering, enables to learn a representation tailored to specific clustering tasks, leading to high-quality results. This survey provides an introduction to fundamental autoencoder-based deep clustering algorithms that serve as building blocks for many modern approaches.

Paper Structure

This paper contains 28 sections, 25 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Autoencoders
  4. Clustering
  5. Autoencoder-based Deep Clustering
  6. Sequential Deep Clustering
  7. Autoencoder with $k$-Means (AE+$k$-Means)
  8. Deep Embedding Network (DEN)
  9. Not Too Deep Clustering (N2D)
  10. Deep Density-based Image Clustering (DDC)
  11. Alternating Deep Clustering
  12. Auto-encoder Based Data Clustering (AEC)
  13. Deep Clustering Network (DCN)
  14. Autoencoder Centroid based Deep-Clustering (ACe/DeC)
  15. DipEncoder
  16. ...and 13 more sections

Figures (4)

  • Figure 1: Categorization of the deep clustering landscape. Topics covered in this survey are highlighted in green.
  • Figure 2: A feedforward autoencoder optimizing the reconstruction loss $\mathcal{L}_{rec}$.
  • Figure 3: Autoencoder-based deep clustering optimizing the reconstruction loss ($\mathcal{L}_{rec}$) and the clustering loss ($\mathcal{L}_{clust}$).
  • Figure 4: Impact of the weights $\lambda_1$ and $\lambda_2$ when optimizing $\lambda_1 \mathcal{L}_{rec} + \lambda_2 \mathcal{L}_{clust}$ - PCA plots of the results of AE+$k$-Means (a) and differently parameterized versions of IDEC (b)-(d) regarding MNIST. REC = Reconstruction Loss, SIL = Silhouette Score, ACC = Unsupervised Clustering Accuracy. Colors correspond to the ground truth.