DeepVAT: A Self-Supervised Technique for Cluster Assessment in Image Datasets

TL;DR

DeepVAT tackles the problem of estimating cluster structure in unlabeled, high-dimensional image data by learning cluster-friendly embeddings with a self-supervised SimCLR model, reducing to 2D with $t$-SNE, and applying VAT/iVAT on a smartly sampled subset to infer the number of clusters without prior knowledge of $k$. By integrating deep representation learning with the VAT family, the approach yields sharper RDIs and higher PA/NMI across MNIST, FMNIST, CIFAR-10, and Intel Image Dataset than state-of-the-art VAT variants and two deep clustering baselines. The key contributions include (1) first use of deep, self-supervised features within a VAT framework for images, (2) a Maximin Random Sampling (MMRS) strategy to scale VAT, and (3) a comprehensive experimental validation demonstrating significant improvements in cluster-structure estimation. This framework enables more reliable exploratory data analysis for complex image datasets and suggests future work to reduce training time and explore semi-supervised enhancements to further improve clustering assessments.

Abstract

Estimating the number of clusters and cluster structures in unlabeled, complex, and high-dimensional datasets (like images) is challenging for traditional clustering algorithms. In recent years, a matrix reordering-based algorithm called Visual Assessment of Tendency (VAT), and its variants have attracted many researchers from various domains to estimate the number of clusters and inherent cluster structure present in the data. However, these algorithms face significant challenges when dealing with image data as they fail to effectively capture the crucial features inherent in images. To overcome these limitations, we propose a deep-learning-based framework that enables the assessment of cluster structure in complex image datasets. Our approach utilizes a self-supervised deep neural network to generate representative embeddings for the data. These embeddings are then reduced to 2-dimension using t-distributed Stochastic Neighbour Embedding (t-SNE) and inputted into VAT based algorithms to estimate the underlying cluster structure. Importantly, our framework does not rely on any prior knowledge of the number of clusters. Our proposed approach demonstrates superior performance compared to state-of-the-art VAT family algorithms and two other deep clustering algorithms on four benchmark image datasets, namely MNIST, FMNIST, CIFAR-10, and INTEL.

Figures (2)

• Figure 1: (a) iVAT image of 100-dimensional Compact and Separated (CS) Gaussian mixture data (3 Gaussians); (b) iVAT image of flattened MNIST data (784 dimensional)
• Figure 2: The proposed architecture of DeepVAT