Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NeurCAM: Interpretable Neural Clustering via Additive Models

Nakul Upadhya, Eldan Cohen

TL;DR

NeurCAM addresses the need for interpretable clustering by combining neural generalized additive models with a fuzzy clustering objective, producing additive, human-understandable explanations in the input feature space. It introduces feature and interaction selection gates to enforce sparsity, ensuring concise, modular explanations, and extends to Neur2CAM to capture pairwise interactions. The approach yields performance comparable to black-box methods on tabular data and significantly outperforms other interpretable clustering methods on text data, aided by contextual embeddings like MPNet. This framework provides a practical, scalable path toward trustworthy clustering with interpretable, simulable explanations suitable for knowledge discovery across domains.

Abstract

Interpretable clustering algorithms aim to group similar data points while explaining the obtained groups to support knowledge discovery and pattern recognition tasks. While most approaches to interpretable clustering construct clusters using decision trees, the interpretability of trees often deteriorates on complex problems where large trees are required. In this work, we introduce the Neural Clustering Additive Model (NeurCAM), a novel approach to the interpretable clustering problem that leverages neural generalized additive models to provide fuzzy cluster membership with additive explanations of the obtained clusters. To promote sparsity in our model's explanations, we introduce selection gates that explicitly limit the number of features and pairwise interactions leveraged. Additionally, we demonstrate the capacity of our model to perform text clustering that considers the contextual representation of the texts while providing explanations for the obtained clusters based on uni- or bi-word terms. Extensive experiments show that NeurCAM achieves performance comparable to black-box methods on tabular datasets while remaining interpretable. Additionally, our approach significantly outperforms other interpretable clustering approaches when clustering on text data.

NeurCAM: Interpretable Neural Clustering via Additive Models

TL;DR

NeurCAM addresses the need for interpretable clustering by combining neural generalized additive models with a fuzzy clustering objective, producing additive, human-understandable explanations in the input feature space. It introduces feature and interaction selection gates to enforce sparsity, ensuring concise, modular explanations, and extends to Neur2CAM to capture pairwise interactions. The approach yields performance comparable to black-box methods on tabular data and significantly outperforms other interpretable clustering methods on text data, aided by contextual embeddings like MPNet. This framework provides a practical, scalable path toward trustworthy clustering with interpretable, simulable explanations suitable for knowledge discovery across domains.

Abstract

Interpretable clustering algorithms aim to group similar data points while explaining the obtained groups to support knowledge discovery and pattern recognition tasks. While most approaches to interpretable clustering construct clusters using decision trees, the interpretability of trees often deteriorates on complex problems where large trees are required. In this work, we introduce the Neural Clustering Additive Model (NeurCAM), a novel approach to the interpretable clustering problem that leverages neural generalized additive models to provide fuzzy cluster membership with additive explanations of the obtained clusters. To promote sparsity in our model's explanations, we introduce selection gates that explicitly limit the number of features and pairwise interactions leveraged. Additionally, we demonstrate the capacity of our model to perform text clustering that considers the contextual representation of the texts while providing explanations for the obtained clusters based on uni- or bi-word terms. Extensive experiments show that NeurCAM achieves performance comparable to black-box methods on tabular datasets while remaining interpretable. Additionally, our approach significantly outperforms other interpretable clustering approaches when clustering on text data.
Paper Structure (43 sections, 18 equations, 5 figures, 15 tables, 1 algorithm)

This paper contains 43 sections, 18 equations, 5 figures, 15 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Interpretable Clustering
  3. Interpretable Out-of-Sample Mapping
  4. Model-Based Interpretability
  5. Existing Approaches
  6. Generalized Additive Models
  7. Interpretability of GAMs
  8. NeurCAM
  9. Model Architecture
  10. Neural Basis Model
  11. Feature Selection Gates
  12. Extending to Neur2CAM:
  13. Training NeurCAM
  14. Fuzzy Clustering Loss
  15. Disentangling Representations
  16. ...and 28 more sections

Figures (5)

  • Figure 1: Our proposed approach. We leverage multiple shape functions that each pick a feature via a selection gate. The final prediction is the sum of the individual shape function contributions. The black shape functions represents NeurCAM and the additional blue pairwise shape functions represent Neur2CAM.
  • Figure 2: Cost Ratio of Neur2CAM compared to MKMC as more pairwise shape functions are added.
  • Figure 3: Shape Graphs for "Microsoft" learned when clustering the AG News Dataset
  • Figure 4: Single Feature Shape Graphs for the Shuttle Dataset (Top) as well as feature value histograms (bottom)
  • Figure 5: Pairwise Feature Shape Graph for the Shuttle Dataset