Deep Clustering via Probabilistic Ratio-Cut Optimization
Ayoub Ghriss, Claire Monteleoni
TL;DR
The paper tackles clustering by optimizing the graph ratio-cut without relying on eigendecomposition, by modeling cluster assignments as random variables and learning with online stochastic gradient descent. It introduces PRCut, which augments the probabilistic ratio-cut objective ${\mathcal L}_{rc}$ with a KL regularization term ${D_{KL}}(\overline{P} \| \tfrac{1}{k}\mathbf{1}_k)$ to prevent collapse, and provides an unbiased gradient estimator based on online estimates of the mean assignment. Empirically, PRCut yields a tighter ratio-cut objective and competitive or superior clustering performance on MNIST, Fashion-MNIST, and CIFAR using self-supervised representations, approaching supervised-classifier quality when similarity is label-based. The approach offers a scalable, non-spectral route to leveraging modern representations for clustering and can serve as a tool for evaluating representation quality and pre-training pipelines, with potential extensions to offline learning and dynamic cluster counts.
Abstract
We propose a novel approach for optimizing the graph ratio-cut by modeling the binary assignments as random variables. We provide an upper bound on the expected ratio-cut, as well as an unbiased estimate of its gradient, to learn the parameters of the assignment variables in an online setting. The clustering resulting from our probabilistic approach (PRCut) outperforms the Rayleigh quotient relaxation of the combinatorial problem, its online learning extensions, and several widely used methods. We demonstrate that the PRCut clustering closely aligns with the similarity measure and can perform as well as a supervised classifier when label-based similarities are provided. This novel approach can leverage out-of-the-box self-supervised representations to achieve competitive performance and serve as an evaluation method for the quality of these representations.