Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Embedding Clustering Driven by Sample Stability

Zhanwen Cheng, Feijiang Li, Jieting Wang, Yuhua Qian

TL;DR

This work tackles the bias-inducing reliance on pseudo targets in deep clustering by introducing Deep Embedding Clustering Driven by Sample Stability (DECS). DECS combines a convolutional autoencoder for representation learning with a clustering objective guided by sample stability, using determinacy-based metrics and a stability loss $L_c$ to steer embeddings toward cluster structure while avoiding pseudo-labels. The approach yields a joint optimization of reconstruction and stability-driven clustering, with a Lipschitz-continuity–based convergence analysis and state-of-the-art results on five image datasets (ACC and NMI). By replacing pseudo targets with stability constraints, DECS reduces clustering bias and enhances robustness for high-dimensional data, offering a practical unsupervised clustering paradigm for complex visual data.

Abstract

Deep clustering methods improve the performance of clustering tasks by jointly optimizing deep representation learning and clustering. While numerous deep clustering algorithms have been proposed, most of them rely on artificially constructed pseudo targets for performing clustering. This construction process requires some prior knowledge, and it is challenging to determine a suitable pseudo target for clustering. To address this issue, we propose a deep embedding clustering algorithm driven by sample stability (DECS), which eliminates the requirement of pseudo targets. Specifically, we start by constructing the initial feature space with an autoencoder and then learn the cluster-oriented embedding feature constrained by sample stability. The sample stability aims to explore the deterministic relationship between samples and all cluster centroids, pulling samples to their respective clusters and keeping them away from other clusters with high determinacy. We analyzed the convergence of the loss using Lipschitz continuity in theory, which verifies the validity of the model. The experimental results on five datasets illustrate that the proposed method achieves superior performance compared to state-of-the-art clustering approaches.

Deep Embedding Clustering Driven by Sample Stability

TL;DR

This work tackles the bias-inducing reliance on pseudo targets in deep clustering by introducing Deep Embedding Clustering Driven by Sample Stability (DECS). DECS combines a convolutional autoencoder for representation learning with a clustering objective guided by sample stability, using determinacy-based metrics and a stability loss to steer embeddings toward cluster structure while avoiding pseudo-labels. The approach yields a joint optimization of reconstruction and stability-driven clustering, with a Lipschitz-continuity–based convergence analysis and state-of-the-art results on five image datasets (ACC and NMI). By replacing pseudo targets with stability constraints, DECS reduces clustering bias and enhances robustness for high-dimensional data, offering a practical unsupervised clustering paradigm for complex visual data.

Abstract

Deep clustering methods improve the performance of clustering tasks by jointly optimizing deep representation learning and clustering. While numerous deep clustering algorithms have been proposed, most of them rely on artificially constructed pseudo targets for performing clustering. This construction process requires some prior knowledge, and it is challenging to determine a suitable pseudo target for clustering. To address this issue, we propose a deep embedding clustering algorithm driven by sample stability (DECS), which eliminates the requirement of pseudo targets. Specifically, we start by constructing the initial feature space with an autoencoder and then learn the cluster-oriented embedding feature constrained by sample stability. The sample stability aims to explore the deterministic relationship between samples and all cluster centroids, pulling samples to their respective clusters and keeping them away from other clusters with high determinacy. We analyzed the convergence of the loss using Lipschitz continuity in theory, which verifies the validity of the model. The experimental results on five datasets illustrate that the proposed method achieves superior performance compared to state-of-the-art clustering approaches.
Paper Structure (16 sections, 2 theorems, 23 equations, 5 figures, 2 tables, 1 algorithm)

This paper contains 16 sections, 2 theorems, 23 equations, 5 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Convolutional Autoencoder
  4. Sample Stability
  5. Method
  6. Problem Formulation
  7. Extract Features with Convolutional Autoencoder
  8. Clustering with Sample Stability
  9. Optimization and Convergence Analysis
  10. Experiments
  11. Datasets and Evaluation Metrics
  12. Baseline Methods
  13. Experiment Results
  14. Visualization Analyses
  15. Implementation
  16. ...and 1 more sections

Key Result

Theorem 1

There exists $M > 0$ such that $||\bigtriangledown L_{c}|| \le M$, where $M = \frac{2\left ( 1+2\lambda \right ) \left ( \alpha +1 \right ) }{4nkt^{2} \alpha } max(\left| \left| z_{i}-m_{j} \right| \right|)$.

Figures (5)

  • Figure 1: Pipeline of the proposed DECS. We first train an autoencoder consisting of an encoder and a decoder to embed the inputs into a latent space and reconstruct the input samples using their latent representations. The reconstruction loss is utilized to learn discriminative information from the inputs. Then, we discard the decoder and jointly optimize the encoder and clustering model to get the clustering results.
  • Figure 2: Visualization of the functions and their derivatives involved in sample stability clustering. (a) shows the function of $sq$ w.r.t $fq$ and its derivative in the case of two centroids. (b) represents the function of $fq$ w.r.t $q$ and its derivative. (c) demonstrates the function of $q$ w.r.t two dimensions vector $z$ and its derivative. (d) depicts the function of $q$ w.r.t two cluster centers $m$ and its derivative.
  • Figure 3: T-SNE visualization comparing the cluster structures obtained from different clustering algorithms on the USPS datasets. Distinct colors represent different digits, and the cluster centers are indicated by black 'x' symbols.
  • Figure 4: ACC and NMI of our method with different $\alpha$ and $\lambda$ on USPS dataset.
  • Figure 5: T-SNE visualization on the learned USPS representations across the training process. Different digits are denoted with different colors, and the black symbol "x" denotes the cluster centers estimated by DECS.

Theorems & Definitions (3)

  • Theorem 1
  • proof
  • Lemma 1