EPIC: Explanation of Pretrained Image Classification Networks via Prototype

TL;DR

The paper tackles the interpretability of pretrained image classifiers by bridging post-hoc explanations and prototype-based, inherently interpretable methods. It introduces EPIC, a plug-in Disentanglement Module that learns an invertible $U \in \mathbb{R}^{D\times D}$ to disentangle final-layer channels without retraining the backbone, preserving the original logits via $A' = A U^{-1}$. Prototype purity is defined as $\mathrm{purity}(I,k) = \frac{p_k}{\|p\|}$ and drives the selection of channel prototypes, yielding intuitive, part-based explanations. Empirical results on CUB-200-2011, Stanford Cars, and ImageNet show EPIC delivers higher interpretability than saliency maps while maintaining competitive classification accuracy, supported by multidimensional analyses and user studies.

Abstract

Explainable AI (XAI) methods generally fall into two categories. Post-hoc approaches generate explanations for pre-trained models and are compatible with various neural network architectures. These methods often use feature importance visualizations, such as saliency maps, to indicate which input regions influenced the model's prediction. Unfortunately, they typically offer a coarse understanding of the model's decision-making process. In contrast, ante-hoc (inherently explainable) methods rely on specially designed model architectures trained from scratch. A notable subclass of these methods provides explanations through prototypes, representative patches extracted from the training data. However, prototype-based approaches have limitations: they require dedicated architectures, involve specialized training procedures, and perform well only on specific datasets. In this work, we propose EPIC (Explanation of Pretrained Image Classification), a novel approach that bridges the gap between these two paradigms. Like post-hoc methods, EPIC operates on pre-trained models without architectural modifications. Simultaneously, it delivers intuitive, prototype-based explanations inspired by ante-hoc techniques. To the best of our knowledge, EPIC is the first post-hoc method capable of fully replicating the core explanatory power of inherently interpretable models. We evaluate EPIC on benchmark datasets commonly used in prototype-based explanations, such as CUB-200-2011 and Stanford Cars, alongside large-scale datasets like ImageNet, typically employed by post-hoc methods. EPIC uses prototypes to explain model decisions, providing a flexible and easy-to-understand tool for creating clear, high-quality explanations.

Figures (13)

• Figure 1: Comparison of explanations constructed by EPIC, and classical post-hoc models: Grad-CAM and LRP. The experiment is presented in the ResNet50 feature space on the Cactus Wren image from the CUB200-2011 dataset. Each row of EPIC (our) represents the prototypical part. The yellow boxes in each row show the activation of a given prototypical part, while in the first column, we show the activation of corresponding prototypical parts in the original image. Observe that contrary to the classical XAI post-hoc approaches (Grad-CAM and LRP), EPIC provides an explanation behind the decision of the model.
• Figure 2: Our image classification interpretation model, EPIC, features three main components: a pre-trained backbone, a disentanglement layer for key features, and a fully connected layer. In contrast to the classical model, we introduce a square matrix of size equal to the number of channels, which enables disentanglement of key features. To ensure the logits remain comparable to those of the classical model, we modify the weights in the fully connected layer by multiplying them with the inverse transformation used in the feature disentanglement step.
• Figure 3: Explanations for a blue jay bird, before (left) and after (right) EPIC training on Resnet18. As we can see, prototypes without additional tuning correspond to random images and have limited explanatory properties. After EPIC tuning, such prototypes are consistent and correspond with input image features.
• Figure 4: The illustration demonstrates the concept of the Purity of Prototype mechanism. For a selected channel, the vector $\mathbf{z}$ (shown on the left) is defined by the maximum pixel value in that channel, making its values comparable (histogram of activation is flat). After optimizing the purity of the given prototype, only one dominant value remains in the refined vector $\mathbf{\tilde{z}}$, as seen on the right. Repeating this process for each channel results in a disentangled representation, where each channel contains only one dominant value associated with its prototype.
• Figure 5: Explanations for the Hognose Snake from ImageNet constructed by EPIC (our), Grad-CAM and LRP. EPIC effectively capture crucial concepts, such as shapes, colors, textures, and distinctive features like the snake's eye area. In contrast, Grad-CAM and LRP produce only saliency maps, offering less interpretability regarding specific visual attributes and concepts.

Theorems & Definitions (2)

• Remark 3.1
• proof