Towards Human-Understandable Multi-Dimensional Concept Discovery

TL;DR

HU-MCD tackles the problem of making concept-based explanations both human-understandable and faithful to the model's decisions. It introduces SAM-based concept discovery to identify semantic regions and a CNN-tailored input masking scheme to reduce masking artifacts, all within the Multi-Dimensional Concept Discovery framework to preserve completeness. The approach yields local and global concept relevance scores that reconstruct model outputs and is validated through a large human study and C-Deletion/C-Insertion benchmarks on ImageNet1k, outperforming ACE and MCD in understandability and faithfulness. This work advances practical eXplainable AI by enabling interpretable, completeness-aware explanations suitable for real-world deployment and regulatory compliance.

Abstract

Concept-based eXplainable AI (C-XAI) aims to overcome the limitations of traditional saliency maps by converting pixels into human-understandable concepts that are consistent across an entire dataset. A crucial aspect of C-XAI is completeness, which measures how well a set of concepts explains a model's decisions. Among C-XAI methods, Multi-Dimensional Concept Discovery (MCD) effectively improves completeness by breaking down the CNN latent space into distinct and interpretable concept subspaces. However, MCD's explanations can be difficult for humans to understand, raising concerns about their practical utility. To address this, we propose Human-Understandable Multi-dimensional Concept Discovery (HU-MCD). HU-MCD uses the Segment Anything Model for concept identification and implements a CNN-specific input masking technique to reduce noise introduced by traditional masking methods. These changes to MCD, paired with the completeness relation, enable HU-MCD to enhance concept understandability while maintaining explanation faithfulness. Our experiments, including human subject studies, show that HU-MCD provides more precise and reliable explanations than existing C-XAI methods. The code is available at https://github.com/grobruegge/hu-mcd.

Figures (7)

• Figure 1: Overview of HU-MCD.
• Figure 2: Survey sample of the human subject experiment generated by HU-MCD. Participants are asked to assign the test image on the left to the group on the right which is most similar by only considering the highlighted region.
• Figure 3: Concept examples for three of the ten CIFAR-10 alike classes generated by HU-MCD.
• Figure 4: We delete (left) or insert (right) concepts in decreasing order of concept importance and measure the impact on model prediction accuracy, averaged over all validation images of ten ImageNet1k classes. Each point represents a discovered concept. Faithful concept importance scores are supposed to result in a sharp decline (left) or ascent (right).
• Figure 5: Example of outputs shown to participants: left ACE, middle MCD, right HU-MCD. Accuracy of descriptions can be found in Section 4.