Explaning with trees: interpreting CNNs using hierarchies

TL;DR

This paper introduces xAiTrees, a hierarchical-region explainability framework for CNNs that balances fidelity to model reasoning with human interpretability. It leverages multiscale segmentation (via Binary Partition Trees and Hierarchical Watershed) and both human-based and model-based region attributions to produce explanations, further enhanced by a persistence-based tree shaping and aggregation into a final visualization. A novel Pixel Impact Rate (PIR) metric is proposed to quantify per-pixel explanatory power, and extensive experiments across Cat-vs-Dog, CIFAR-10, and ImageNet demonstrate competitive fidelity with strong bias-detection performance, outperforming several baselines. The work includes a human-subject study validating interpretability gains and provides open-source code to encourage adoption and reproducibility.

Abstract

Challenges persist in providing interpretable explanations for neural network reasoning in explainable AI (xAI). Existing methods like Integrated Gradients produce noisy maps, and LIME, while intuitive, may deviate from the model's reasoning. We introduce a framework that uses hierarchical segmentation techniques for faithful and interpretable explanations of Convolutional Neural Networks (CNNs). Our method constructs model-based hierarchical segmentations that maintain the model's reasoning fidelity and allows both human-centric and model-centric segmentation. This approach offers multiscale explanations, aiding bias identification and enhancing understanding of neural network decision-making. Experiments show that our framework, xAiTrees, delivers highly interpretable and faithful model explanations, not only surpassing traditional xAI methods but shedding new light on a novel approach to enhancing xAI interpretability.

Figures (16)

• Figure 1: One challenge in xAI is achieving a good trade-off between fidelity and interpretability. We propose using region-based segmentation combined with hierarchies to adapt region size, while providing the flexibility to use high-fidelity methods for constructing the segmented regions.
• Figure 2: Explanations of six image classes misclassified by VGG-16 or Resnet18 models trained on ImageNet. We compare well-known xAI method explanations with one configuration of xAiTrees: Tree-Occ. Methods such as Integrated Gradients are noisy and difficult to interpret. Shapes such as the grades and the fence seem to be better highlighted by Tree-Occ, which is helpful for interpretation. When compared to highly interpretable methods like LIME, Tree-Occ avoids the mistake of highlighting the cat when the models predict classes such as dishwasher, saltshaker, and hamper.
• Figure 3: Our framework xAiTrees operates through four key steps: 1. Generate a segmentation hierarchy using either the image's edge map for human-based segmentation or pixel-wise importance based on xAI techniques for model-based segmentation. 2. Systematically occlude each region of the segmentation to evaluate its impact on the model's decision, obtaining an occlusion attribute for each region. 3. Assess the persistence of the occlusion attribute using a shaping approach xu2015connectedxu:pami:2016. 4. Aggregate the contributions of each region from the highest to the lowest level of the tree to create a comprehensive multiscale visualization.
• Figure 4: The hierarchy aids to discriminate similar important regions. Example of the method's behavior with the same structure inside and outside a hierarchy. The cat's eyes were replicated outside the cat's face. However, the importance of each region is the combination of the importance of each hierarchy part. Therefore, the cat's eyes inside the face (an important hierarchy region) score higher, as evidenced by the lighter regions in the right image.
• Figure 5: Our quantitative experiments aim to answer the question: Is the explanation faithful? In (1), we occlude the selected important regions and evaluate the impact using the Pixel Impact Rate (PIR) based on the model’s response. In (2), we evaluate the inclusion of only the important regions by analyzing their Softmax and Accuracy Information Curves.