Explaining Black-Box Clustering Pipelines With Cluster-Explorer

TL;DR

Cluster-Explorer presents a universal, post-hoc explainability framework for black-box clustering pipelines by mining concise conjunctions of predicates that describe each cluster. It achieves this via a reduction to generalized frequent itemsets mining on augmented transactions derived from multiple numeric binning and categorical negations, coupled with an interval taxonomy and a skyline-based Pareto optimal selection. An attribute-selection optimization based on decision-tree importance dramatically reduces item-space, yielding substantial speedups without sacrificing explanation quality. Empirical evaluation on 98 clustering results and a user study demonstrates superior explanation quality and faster runtimes compared with XAI baselines, validating practical utility for interpretable cluster analysis.

Abstract

Explaining the results of clustering pipelines by unraveling the characteristics of each cluster is a challenging task, often addressed manually through visualizations and queries. Existing solutions from the domain of Explainable Artificial Intelligence (XAI) are largely ineffective for cluster explanations, and interpretable-by-design clustering algorithms may be unsuitable when the clustering algorithm does not fit the data properties. To bridge this gap, we introduce Cluster-Explorer, a novel explainability tool for black-box clustering pipelines. Our approach formulates the explanation of clusters as the identification of concise conjunctions of predicates that maximize the coverage of the cluster's data points while minimizing separation from other clusters. We achieve this by reducing the problem to generalized frequent-itemsets mining (gFIM), where items correspond to explanation predicates, and itemset frequency indicates coverage. To enhance efficiency, we leverage inherent problem properties and implement attribute selection to further reduce computational costs. Experimental evaluations on a benchmark collection of 98 clustering results, as well as a user study, demonstrate the superiority of Cluster-Explorer in both explanation quality and execution times compared to XAI baselines.

Figures (10)

• Figure 1: Clustering results visualization (Adult dataset)
• Figure 2: Example cluster explanations generated by Cluster-Explorer
• Figure 3: Example augmented transactions
• Figure 4: Combined figures illustrating QSE and metrics analysis.
• Figure 5: User Study Results

Theorems & Definitions (1)

• definition 1: Cluster Explanations Generation Problem