Deep Clustering with Associative Memories

TL;DR

DCAM presents a novel energy-based deep clustering framework that integrates Dense Associative Memories into latent-space clustering. By applying attractor dynamics to latent encodings and jointly updating encoder, decoder, and cluster prototypes, the method yields cluster-friendly representations while preserving reconstruction quality in a fully differentiable pipeline. Empirical results across eight diverse datasets demonstrate superior clustering performance, as measured by silhouette scores and often by NMI, across multiple autoencoder architectures. The approach offers an architecture-agnostic, end-to-end differentiable tool for robust unsupervised clustering with potential extensions to multimodal data and automatic cluster number estimation.

Abstract

Deep clustering - joint representation learning and latent space clustering - is a well studied problem especially in computer vision and text processing under the deep learning framework. While the representation learning is generally differentiable, clustering is an inherently discrete optimization task, requiring various approximations and regularizations to fit in a standard differentiable pipeline. This leads to a somewhat disjointed representation learning and clustering. In this work, we propose a novel loss function utilizing energy-based dynamics via Associative Memories to formulate a new deep clustering method, DCAM, which ties together the representation learning and clustering aspects more intricately in a single objective. Our experiments showcase the advantage of DCAM, producing improved clustering quality for various architecture choices (convolutional, residual or fully-connected) and data modalities (images or text).

Figures (12)

• Figure 1: Clustering with DCAM. Reconstruction loss (RL; refers to the loss $\bar{{\mathcal{L}}}$ in \ref{['eq:dc-am']}) and clustering quality (SC; refers to Silhouette Coefficient) for different epochs (e) and number of AM steps (t) for Fashion-MNIST with two clusters, where \ref{['fig:hp:pretrained']} represents the pretrained latent representations. The light (faded) colors indicate the encoded points before applying the attractor dynamics, whereas dark colors indicate the points after applying that step. The colored stars represent the learned prototypes. DCAM discovers more compact and clustering-friendly latent representations that simultaneously have higher clustering and reconstruction quality.
• Figure 2: DCAM: AM-enabled deep clustering. The solid arrows $\boldsymbol{\xrightarrow{\ \ \ \ \ \ }}$ denote the forward-pass to compute the single loss term in \ref{['eq:dc-am']}. The dashed arrows $\boldsymbol{\dashrightarrow}$ denote the backward-pass showing the single loss driving all updates.
• Figure 3: Visualizing the decoded images for the learned cluster prototypes (leftmost column in block) and the corresponding closest (center column in block) and farthest (right column in block) members in each cluster for Fashion MNIST (left block) and Caltech Birds (right block).
• Figure 4: Reconstruction loss (RL) and clustering quality (SC) for varying number of steps (T) for FMNIST. Red point in \ref{['fig:st:fm-st-rl']} indicates the lowest reconstruction loss and orange points indicate the reconstruction loss within 10% of this lowest reconstruction loss.
• Figure 5: Evolution of prototypes for Fashion-MNIST and USPS for DCAM. We visualize the prototypes at the $n^{\text{th}}$ training epoch for $n = 0, 5, 10, 20, 50, 100$ (with $T=10$).