Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DiEC: Diffusion Embedded Clustering

Haidong Hu

TL;DR

DiEC tackles unsupervised clustering by leveraging internal activations from a pretrained diffusion U-Net, treating the layer-by-timestep activation trajectory as a rich clustering-relevant representation. It identifies a clustering-friendly location by fixing the Clustering Middle Layer (CML) at the U-Net bottleneck and efficiently localizing the clustering-optimal timestep $t^{*}$ with Optimal Timestep Search (OTS), then learns clustering representations via a residual mapping and a DEC-style KL objective augmented with adaptive graph and entropy regularization. A diffusion-consistency branch at random timesteps stabilizes training and preserves generative behavior. Empirical results on MNIST, USPS, Fashion-MNIST, and CIFAR-10 demonstrate competitive clustering performance, highlighting the practical value of exploiting diffusion-based internal representations for unsupervised learning.

Abstract

Deep clustering hinges on learning representations that are inherently clusterable. However, using a single encoder to produce a fixed embedding ignores the representation trajectory formed by a pretrained diffusion model across network hierarchies and noise timesteps, where clusterability varies substantially. We propose DiEC (Diffusion Embedded Clustering), which performs unsupervised clustering by directly reading internal activations from a pretrained diffusion U-Net. DiEC formulates representation selection as a two-dimensional search over layer x timestep, and exploits a weak-coupling property to decompose it into two stages. Specifically, we first fix the U-Net bottleneck layer as the Clustering-friendly Middle Layer (CML), and then use Optimal Timestep Search (OTS) to identify the clustering-optimal timestep (t*). During training, we extract bottleneck features at the fixed t* and obtain clustering representations via a lightweight residual mapping. We optimize a DEC-style KL self-training objective, augmented with adaptive graph regularization and entropy regularization to strengthen cluster structures. In parallel, we introduce a denoising-consistency branch at random timesteps to stabilize the representations and preserve generative consistency. Experiments show that DiEC achieves competitive clustering performance on multiple standard benchmarks.

DiEC: Diffusion Embedded Clustering

TL;DR

DiEC tackles unsupervised clustering by leveraging internal activations from a pretrained diffusion U-Net, treating the layer-by-timestep activation trajectory as a rich clustering-relevant representation. It identifies a clustering-friendly location by fixing the Clustering Middle Layer (CML) at the U-Net bottleneck and efficiently localizing the clustering-optimal timestep with Optimal Timestep Search (OTS), then learns clustering representations via a residual mapping and a DEC-style KL objective augmented with adaptive graph and entropy regularization. A diffusion-consistency branch at random timesteps stabilizes training and preserves generative behavior. Empirical results on MNIST, USPS, Fashion-MNIST, and CIFAR-10 demonstrate competitive clustering performance, highlighting the practical value of exploiting diffusion-based internal representations for unsupervised learning.

Abstract

Deep clustering hinges on learning representations that are inherently clusterable. However, using a single encoder to produce a fixed embedding ignores the representation trajectory formed by a pretrained diffusion model across network hierarchies and noise timesteps, where clusterability varies substantially. We propose DiEC (Diffusion Embedded Clustering), which performs unsupervised clustering by directly reading internal activations from a pretrained diffusion U-Net. DiEC formulates representation selection as a two-dimensional search over layer x timestep, and exploits a weak-coupling property to decompose it into two stages. Specifically, we first fix the U-Net bottleneck layer as the Clustering-friendly Middle Layer (CML), and then use Optimal Timestep Search (OTS) to identify the clustering-optimal timestep (t*). During training, we extract bottleneck features at the fixed t* and obtain clustering representations via a lightweight residual mapping. We optimize a DEC-style KL self-training objective, augmented with adaptive graph regularization and entropy regularization to strengthen cluster structures. In parallel, we introduce a denoising-consistency branch at random timesteps to stabilize the representations and preserve generative consistency. Experiments show that DiEC achieves competitive clustering performance on multiple standard benchmarks.
Paper Structure (12 sections, 25 equations, 1 figure, 2 tables)

This paper contains 12 sections, 25 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Clustering in Learned Embedding Spaces.
  4. Diffusion Models as Feature Extractors.
  5. Diffusion for Clustering.
  6. Background
  7. Diffusion Embedded clustering
  8. Locating Clustering-Friendly Diffusion Representations
  9. Clustering-Aware Optimization
  10. Loss Function
  11. Experiment
  12. Conclusion

Figures (1)

  • Figure 1: Overview of DiEC (Diffusion Embedded Clustering). Given an input image $x_0$, we inject noise using both a random timestep $t_{\mathrm{rand}}$ and a fixed clustering-optimal timestep (COT). The random-timestep branch preserves generative consistency via standard denoising, while the fixed-timestep branch extracts the Clustering Middle Layer(CML) for clustering-aware optimization. We apply residual decoupling on CML and jointly optimize a graph regularizer and a KL-based clustering objective to obtain more stable and discriminative clustering representations.