Occlusion Sensitivity Analysis with Augmentation Subspace Perturbation in Deep Feature Space

TL;DR

This work addresses the challenge of explainability in vision models by introducing Occlusion Sensitivity Analysis with Deep Feature Augmentation Subspaces (OSA-DAS). The method augments traditional occlusion analysis by generating multiple augmented views and representing both the reference and occluded inputs as low-dimensional subspaces in the deep feature space, then quantifying occlusion impact via subspace distance using canonical angles. Key contributions include a data-augmentation–driven subspace framework, a formalized orthogonal-degree distance for attribution, a gradient-informed masking speedup, and a Minimal Size plus Overall performance metric to evaluate explanations. Experiments on ImageNet-1k across ResNet-50, ViT-B, and Swin-V2 show that OSA-DAS yields more robust, class-agnostic explanations with competitive or superior overall metrics compared to standard interpreters, albeit with higher memory requirements. This approach enhances trust in AI systems by tying explanations to training-time augmentations and deep feature representations, offering a scalable, architecture-agnostic path to reliable model interpretability.

Abstract

Deep Learning of neural networks has gained prominence in multiple life-critical applications like medical diagnoses and autonomous vehicle accident investigations. However, concerns about model transparency and biases persist. Explainable methods are viewed as the solution to address these challenges. In this study, we introduce the Occlusion Sensitivity Analysis with Deep Feature Augmentation Subspace (OSA-DAS), a novel perturbation-based interpretability approach for computer vision. While traditional perturbation methods make only use of occlusions to explain the model predictions, OSA-DAS extends standard occlusion sensitivity analysis by enabling the integration with diverse image augmentations. Distinctly, our method utilizes the output vector of a DNN to build low-dimensional subspaces within the deep feature vector space, offering a more precise explanation of the model prediction. The structural similarity between these subspaces encompasses the influence of diverse augmentations and occlusions. We test extensively on the ImageNet-1k, and our class- and model-agnostic approach outperforms commonly used interpreters, setting it apart in the realm of explainable AI.

Figures (6)

• Figure 1: OSA-DAS Overview: Subspace $\mathcal{V}$ is derived from the augmented input image, while $\mathcal{V_M}$ originates from its occluded counterpart. Both are derived from the principal component analysis (PCA) of a DNN's deep feature vector. The orthogonal degree]fukui_difference_2015fukui_subspace_2020 between $\mathcal{V}$ and $\mathcal{V_M}$ quantifies the occlusion's effect and shapes the explanation heatmap. Multiple occlusion augmentation subspaces are used to capture diverse facets of the input's representation. Their combined relationships offer a holistic view of occlusion impacts, producing a detailed heatmap.
• Figure 2: Mask anchor point selection via gradient sampling. The image gradient is produced on inference time, which is then used to sample anchor points. Anchor points too close to each other are filtered out. (anchors size is increased for visibility)
• Figure 3: Explanation heatmaps visualizations for ResNet-50. Regions in red indicate the prediction causes. The proposed method generate concise and smooth explanation heatmaps, more in line to the general features the model is attending than other techniques.
• Figure 4: Simplified schema for computing the minimal size on a image-heatmap pair of $224 \times 224$ pixels with tolerance $\delta = 10^{-2}$. This simplified version decreases the iterations as follows: First, we divide the explanation heatmap into regions with the same importance level according to a contour map. Then, we introduce pixels from each region to the partial image in descending order of importance. We stop when the model's output of this partial image becomes very close to the one of the original image. The fraction of filled pixels in the partial image is the minimal size metric.
• Figure 5: Explanation heatmaps visualizations for ViT-B and Swin-V2. Regions in red indicate the prediction causes.

Theorems & Definitions (28)

• Definition 6.1: Singleton cause
• Definition 6.2: Cause witness
• Definition 6.3: Simplified Degree of Responsibility
• Definition 6.4: Explanation
• Remark 1: Triviality
• Remark 2: Non-uniqueness
• Definition 6.5: Approximate Explanation
• Remark 3: Normalization
• Remark 4: Approximate Explanation and Degree of responsibility
• Remark 5: Composition