Activation-Deactivation: A General Framework for Robust Post-hoc Explainable AI

TL;DR

This work addresses the fragility of occlusion-based post-hoc explanations, which can produce out-of-distribution inputs and depend on chosen occlusion values. It introduces Activation-Deactivation (AD), a forward-pass paradigm that deactivates model activations corresponding to occluded input regions, and ConvAD, a drop-in CNN module that implements AD without retraining. By grounding AD in depth-2 causal models and restrictions, the approach yields explanations that are robust across contexts while preserving the original model’s decisions on unoccluded inputs. Empirical results across multiple CNN architectures and datasets show that ConvAD explanations outperform traditional masking in robustness (up to 62.5% improvements) and are competitive in size and confidence, supporting practical adoption in real-world explainability workflows.

Abstract

Black-box explainability methods are popular tools for explaining the decisions of image classifiers. A major drawback of these tools is their reliance on mutants obtained by occluding parts of the input, leading to out-of-distribution images. This raises doubts about the quality of the explanations. Moreover, choosing an appropriate occlusion value often requires domain knowledge. In this paper we introduce a novel forward-pass paradigm Activation-Deactivation (AD), which removes the effects of occluded input features from the model's decision-making by switching off the parts of the model that correspond to the occlusions. We introduce ConvAD, a drop-in mechanism that can be easily added to any trained Convolutional Neural Network (CNN), and which implements the AD paradigm. This leads to more robust explanations without any additional training or fine-tuning. We prove that the ConvAD mechanism does not change the decision-making process of the network. We provide experimental evaluation across several datasets and model architectures. We compare the quality of AD-explanations with explanations achieved using a set of masking values, using the proxies of robustness, size, and confidence drop-off. We observe a consistent improvement in robustness of AD explanations (up to 62.5%) compared to explanations obtained with occlusions, demonstrating that ConvAD extracts more robust explanations without the need for domain knowledge.

Figures (18)

• Figure 1: An image of an ibex (a) and its explanations using our method AD (b) compared to occlusion-based methods with different occlusion values (c)-(f) on the EfficientNet-v2 model.
• Figure 2: ① The state-of-the-art approach to generating perturbations for post hoc explainability: masking parts of the input image. ② Our approach Activation-Deactivation (AD): occluding parts of the model. AD preserves the spatial locality of the unmasked features while removing any consideration of the masked features, remaining in distribution for the input features to the model.
• Figure 3: Visual demonstration of applying the position attribution function, in the highlighted section we have two unmasked values, leading to an attribution of 0.5.
• Figure 4: Scenarios where leaks can occur when applying ConvAD.
• Figure 5: $\rho$-robustness (rows, from $0$ to $1$ in steps of $0.2$) of explanation against non-informative background on ImageNet-1k, ImageNet-1k_v2, CalTech(CT)-256 and PASCAL-VOC for our models, with different confidence thresholds $\gamma$. AD explanations are consistently more robust than ones computed using masking values, for all masking values and with all confidence thresholds.

Theorems & Definitions (11)

• Definition 1: Causal explanation for depth-$2$ models
• Definition 2
• Definition 3: Position function ($\mathit{pos}_i$)
• Definition 4: Position-Attribution function ($\Phi_i$)
• Definition 5: Activation-Deactivation (AD)
• Definition 6: Restricted depth-$2$ causal model
• Definition 7: AD-explanation for depth-$2$ models
• Lemma 3.1
• Corollary 3.2
• Definition 8: Actual cause