Graph Cut-guided Maximal Coding Rate Reduction for Learning Image Embedding and Clustering

TL;DR

The paper tackles unsupervised image clustering with pre-trained representations by proposing Graph Cut-guided Maximal Coding Rate Reduction (CgMCR^2), a joint framework that couples a coding-rate based representation objective with a differentiable, graph-cut–guided clustering module. It integrates a CLIP-derived pre-feature extractor, a lightweight feature head, and a Gumbel-Softmax cluster head into a two-stage training procedure (initialization followed by fine-tuning) to produce structured embeddings and accurate cluster memberships. The approach leverages a cosine-based affinity for differentiable spectral-clustering–style optimization and a normalized-cut–based regularization to guide partition learning. Extensive experiments on standard and out-of-domain datasets show state-of-the-art clustering performance and robust ablations validate the contributions of the joint optimization, the clustering module, and the training strategy, with code released for reproducibility.

Abstract

In the era of pre-trained models, image clustering task is usually addressed by two relevant stages: a) to produce features from pre-trained vision models; and b) to find clusters from the pre-trained features. However, these two stages are often considered separately or learned by different paradigms, leading to suboptimal clustering performance. In this paper, we propose a unified framework, termed graph Cut-guided Maximal Coding Rate Reduction (CgMCR$^2$), for jointly learning the structured embeddings and the clustering. To be specific, we attempt to integrate an efficient clustering module into the principled framework for learning structured representation, in which the clustering module is used to provide partition information to guide the cluster-wise compression and the learned embeddings is aligned to desired geometric structures in turn to help for yielding more accurate partitions. We conduct extensive experiments on both standard and out-of-domain image datasets and experimental results validate the effectiveness of our approach.

Figures (9)

• Figure 1: Illustration for our CgMCR$^2$ architecture. We illustrate the forward pass (in black ) and the gradient dependency of $\mathbf{Z}_\mathrm{\Theta}$ (in green ) and $\mathbf{\Pi}_\mathrm{\Phi}$ (in orange ) in different colored lines individually. For clarity, we exclude the parameters of the pre-feature layer from the visualization.
• Figure 2: Similarity matrices ordered by the ground-truth labels of the CLIP features computed by $|\mathbf{X}^\top \mathbf{X}|$ (in the left panel in blue), representation computed by $|\mathbf{Z}_\mathrm{\Theta}^\top \mathbf{Z}_\mathrm{\Theta}|$ (at the first row in blue) and cluster membership $|\mathbf{\Pi}_\mathrm{\Phi} \mathbf{\Pi}_\mathrm{\Phi}^\top|$ (at the second row in red) of CgMCR$^2$ trained with $\{1,2,5,10,20\}$ epochs on CIFAR-10, where the percentage number in bracket is ACC.
• Figure 3: Learning curves of each loss term, ACC, and NMI during training.
• Figure 4: Effect of hyper-parameters on CIFAR-10 (left) and CIFAR-100 (right).
• Figure 5: Effect of model parameters on CIFAR-10 (left) and CIFAR-100 (right).