CoProNN: Concept-based Prototypical Nearest Neighbors for Explaining Vision Models

TL;DR

CoProNN addresses the need for task-specific explanations in XAI by enabling domain experts to define visual concepts and generate corresponding prototypes with text-to-image models. Explanations are produced by a simple kNN in the frozen model backbone's latent space, comparing test instances to concept prototypes and random image sets. The approach yields competitive performance against TCAV and IBD on coarse-grained tasks and excels on fine-grained datasets like wild bees, while supporting offline evaluation and human-in-the-loop studies. Results from quantitative and qualitative user studies indicate improved human-AI collaboration and trust calibration, and the method is modular and extensible for future models and tasks.

Abstract

Mounting evidence in explainability for artificial intelligence (XAI) research suggests that good explanations should be tailored to individual tasks and should relate to concepts relevant to the task. However, building task specific explanations is time consuming and requires domain expertise which can be difficult to integrate into generic XAI methods. A promising approach towards designing useful task specific explanations with domain experts is based on compositionality of semantic concepts. Here, we present a novel approach that enables domain experts to quickly create concept-based explanations for computer vision tasks intuitively via natural language. Leveraging recent progress in deep generative methods we propose to generate visual concept-based prototypes via text-to-image methods. These prototypes are then used to explain predictions of computer vision models via a simple k-Nearest-Neighbors routine. The modular design of CoProNN is simple to implement, it is straightforward to adapt to novel tasks and allows for replacing the classification and text-to-image models as more powerful models are released. The approach can be evaluated offline against the ground-truth of predefined prototypes that can be easily communicated also to domain experts as they are based on visual concepts. We show that our strategy competes very well with other concept-based XAI approaches on coarse grained image classification tasks and may even outperform those methods on more demanding fine grained tasks. We demonstrate the effectiveness of our method for human-machine collaboration settings in qualitative and quantitative user studies. All code and experimental data can be found in our GitHub $\href{https://github.com/TeodorChiaburu/beexplainable}{repository}$.

Figures (9)

• Figure 1: Overview of the CoProNN framework, illustrated on the subset of wild bee images scraped from iNaturalist. Classifier (blue boxes): images of the five pre-selected species are fed along with their labels assigned by entomologists into a standard DNN. Concept based explanations (red boxes): domain experts define a set of intuitive concepts to discriminate the five species in a hierarchical manner (the decision tree block). These concepts are used as prompts to generate prototype images with Stable Diffusion. The prototype images along with a set of random images are mapped into the latent feature space of the DNN's frozen backbone, where they will be fed into kNN. At inference (bottom row): the DNN classifier predicts a species label and an explanation is retrieved based on the kNN-computed similarities between the extracted features of the test sample and the prototype images in the latent space.
• Figure 2: Examples of prototype images generated by SD along with their (abbreviated) prompts underneath. Prototypes (a) to (e) correspond to the ImageNet animal classes, prototypes (f) to (h) to the iNat wild bees.
• Figure 3: CoProNN finds the relevant concepts for discriminating categories better than alternative XAI methods (TCAV and IBD). Top row (plots (a) to (c)) depicts the five ImageNet animal classes. The expected explanations are: "yellow fur black dots" for cheetah, "elongated chequered" for snake, "orange fur black stripes" for tiger, "chequered shell" for turtle and "white black stripes" for zebra. Bottom row (plots (d) to (f)) refers to the iNat wild bees. The expected explanations are (in shorthand): "orange" and "brown" for A. bicolor, "brown" for A. flavipes, "orange" for A. fulva, "yellow" for B. lucorum, "yellow" and "orange" for B. pratorum. A diamond $\diamond$ marks a zero-value. All the plots display the concept relevance scores averaged per class. An extra note on the normalization of the IBD scores can be found in the Appendix and the corresponding concept contribution scores for the whole classes are given in Fig. \ref{['fig:ibd_cls_scores']}. Our CoProNN method identifies relevant concepts with high certainty in both datasets, which is made clear by the peaks in plots (a) and (d).
• Figure 4: Explanations improve human performance. There is a trend towards higher classification accuracy when subjects are given explanations for the AI's prediction as opposed to only being shown the unexplained model prediction.
• Figure 5: Results of a survey for qualitative evaluation of CoProNN show that subjects generally found the explanations delivered by our model helpful and easy to understand.