Interpretable Deep Clustering for Tabular Data

TL;DR

This work addresses the need for interpretable clustering on high-dimensional tabular data by introducing IDC, a two-stage deep clustering framework that jointly yields cluster assignments and explanations. It combines a local self-supervised gating mechanism that selects informative features per sample with a clustering head and a global gate matrix that provides cluster-level interpretations, enabling sample- and cluster-level explanations without domain-specific augmentations. The method demonstrates strong clustering performance and interpretable results across synthetic data, real biomedical datasets, MNIST-derived tabular forms, and even image-domain data treated as tables, while introducing interpretability metrics such as diversity, faithfulness, uniqueness, and generalizability. IDC also reveals an inductive bias favorable to learning high-frequency components in tabular data, and code is released for reproducibility, with future work focusing on addressing correlated features and clustering scale limitations.

Abstract

Clustering is a fundamental learning task widely used as a first step in data analysis. For example, biologists use cluster assignments to analyze genome sequences, medical records, or images. Since downstream analysis is typically performed at the cluster level, practitioners seek reliable and interpretable clustering models. We propose a new deep-learning framework for general domain tabular data that predicts interpretable cluster assignments at the instance and cluster levels. First, we present a self-supervised procedure to identify the subset of the most informative features from each data point. Then, we design a model that predicts cluster assignments and a gate matrix that provides cluster-level feature selection. Overall, our model provides cluster assignments with an indication of the driving feature for each sample and each cluster. We show that the proposed method can reliably predict cluster assignments in biological, text, image, and physics tabular datasets. Furthermore, using previously proposed metrics, we verify that our model leads to interpretable results at a sample and cluster level. Our code is available at https://github.com/jsvir/idc.

Figures (6)

• Figure 1: Illustration of the proposed model. The first step involves self-supervision for learning a meaningful latent representation and sample-level informative features. During this stage, we optimize the parameters of the Gating Network (green) and the autoencoder (blue) that reconstructs $\hat{x}$ from latent embedding $h$. The gating Network learns a sample-specific sparse gate vector $z$ for input sample $x$ such that $x \odot z$ is sufficient for reconstruction via an autoencoder. We train a clustering head (orange) to predict cluster assignments $\hat{y}$ from the latent embedding $h$ by minimizing the mean cluster coding rate loss (see Eq. \ref{['eq:clusterhead']}). This loss is designed to push clusters apart while making each cluster more compact. The Auxiliary Classifier (yellow) is trained on sparse representations $x \odot z \odot Z^G$ to predict cluster labels and optimizes the cluster level gating matrix $Z^G$.
• Figure 2: Visualization of the synthetic dataset. To separate between clusters, the model should select one of the pairs $\{X[1] , X[2]\}$ or $\{X[1], X[3]\}$ of non-background features.
• Figure 3: Left: Faithfulness plot of the proposed method (green) and $K$-Means+SHAP (blue) on MNIST$_{10K}$ subset. Accuracy drop and feature importance are well correlated for our approach $0.96$ (see green dots) while less correlated for SHAP features with $K$-means clustering $0.79$ (see blue dots). Furthermore, notice that $K$-means accuracy only reaches $53\%$ while our method $83\%$. Right: Features selected by different interpretability model using $\text{MNIST}_{10K}$. The features are learned during clustering training as proposed by our approach (top), features selected by SHAP with $K$-means predictor (KM-SHAP), features obtained from Gradient-SHAP (GradSHAP), and features selected by Integrated Gradients explainer (bottom).
• Figure 4: Spectral properties of the learned predictive function using ALLAML dataset. The model trained with the gating network (IDC) has higher Fourier amplitudes at all frequency levels than without gates (IDC$_{w/o\_gates}$) the baseline (TELL). This suggests that IDC can better handle the inductive bias of tabular data.
• Figure 5: The value of $\frac{\partial}{\partial \mu}\mathbb{E}_Z ||{\hbox{\boldmath $Z$}}||_0 \vert _{\mu=0.5} = \frac{1}{\sqrt{2\pi \sigma^2}} e^{-{\frac{1}{8\sigma^2}}}$ for $\sigma = [0.001, 2]$.