Deep clustering using adversarial net based clustering loss
Kart-Leong Lim
TL;DR
This work addresses the challenge of performing deep clustering without a closed-form JS divergence loss by reframing clustering as an adversarial game in latent space. It introduces DCAN, an encoder–discriminator architecture that approximates the Jensen–Shannon divergence between the latent distribution implied by clustering and the encoder’s latent distribution, enabling end-to-end training. The approach yields competitive results on MNIST, Reuters-10k, and CIFAR-10, with notable gains when using pretrained feature extractors, and provides a theoretical bridge linking ABC, KL, and JS divergences through adversarial training. Overall, DCAN offers a principled, scalable pathway to fuse deep representation learning with clustering objectives via adversarial JS divergence estimation.
Abstract
Deep clustering is a recent deep learning technique which combines deep learning with traditional unsupervised clustering. At the heart of deep clustering is a loss function which penalizes samples for being an outlier from their ground truth cluster centers in the latent space. The probabilistic variant of deep clustering reformulates the loss using KL divergence. Often, the main constraint of deep clustering is the necessity of a closed form loss function to make backpropagation tractable. Inspired by deep clustering and adversarial net, we reformulate deep clustering as an adversarial net over traditional closed form KL divergence. Training deep clustering becomes a task of minimizing the encoder and maximizing the discriminator. At optimality, this method theoretically approaches the JS divergence between the distribution assumption of the encoder and the discriminator. We demonstrated the performance of our proposed method on several well cited datasets such as MNIST, REUTERS10K and CIFAR10, achieving on-par or better performance with some of the state-of-the-art deep clustering methods.