Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generative Kernel Spectral Clustering

David Winant, Sonny Achten, Johan A. K. Suykens

TL;DR

GenKSC addresses interpretability in clustering by integrating kernel spectral clustering with a generative model. It learns a parametric feature map and a decoder while optimizing a spectral clustering objective, augmented by a reconstruction loss and a cosine-based cluster loss that uses simplex cluster codes. The model yields a discriminative, explorable latent space where cluster directions can be traversed to visualize and exaggerate distinguishing features, demonstrated on MNIST012 and FashionMNIST. This approach bridges explainable AI with deep clustering and opens avenues for semi-supervised or supervised extensions.

Abstract

Modern clustering approaches often trade interpretability for performance, particularly in deep learning-based methods. We present Generative Kernel Spectral Clustering (GenKSC), a novel model combining kernel spectral clustering with generative modeling to produce both well-defined clusters and interpretable representations. By augmenting weighted variance maximization with reconstruction and clustering losses, our model creates an explorable latent space where cluster characteristics can be visualized through traversals along cluster directions. Results on MNIST and FashionMNIST datasets demonstrate the model's ability to learn meaningful cluster representations.

Generative Kernel Spectral Clustering

TL;DR

GenKSC addresses interpretability in clustering by integrating kernel spectral clustering with a generative model. It learns a parametric feature map and a decoder while optimizing a spectral clustering objective, augmented by a reconstruction loss and a cosine-based cluster loss that uses simplex cluster codes. The model yields a discriminative, explorable latent space where cluster directions can be traversed to visualize and exaggerate distinguishing features, demonstrated on MNIST012 and FashionMNIST. This approach bridges explainable AI with deep clustering and opens avenues for semi-supervised or supervised extensions.

Abstract

Modern clustering approaches often trade interpretability for performance, particularly in deep learning-based methods. We present Generative Kernel Spectral Clustering (GenKSC), a novel model combining kernel spectral clustering with generative modeling to produce both well-defined clusters and interpretable representations. By augmenting weighted variance maximization with reconstruction and clustering losses, our model creates an explorable latent space where cluster characteristics can be visualized through traversals along cluster directions. Results on MNIST and FashionMNIST datasets demonstrate the model's ability to learn meaningful cluster representations.

Paper Structure

This paper contains 10 sections, 5 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Preliminaries and Related Work
  3. The Generative Kernel Spectral Clustering Model
  4. Spectral Clustering Loss
  5. Augmented Losses
  6. The GenKSC Model
  7. Experiments
  8. Datasets and Model Details
  9. Results
  10. Conclusion

Figures (3)

  • Figure 1: Visualization of latent structure with KSC alzate2008multiway. Left: The data in its original space. Right: The spectral embeddings in the eigenspace of the first two components. Observe that the cluster prototypes align at the tips of the lines. A radial basis function kernel is used in this example.
  • Figure 2: Generated images along indicated cluster directions of the first two dimensions of the latent space for MNIST012.
  • Figure 3: Latent space traversals for the FashionMNIST dataset. Left: The traversals along the cluster directions in the 9-dimensional latent subspace. Right: Traversals along the $k$-th dimension of the latent space.