Identifying Important Group of Pixels using Interactions

TL;DR

This paper introduces MoXI, a learning-friendly, game-theoretic method for explaining image classifications by identifying groups of pixels whose cooperative interactions strongly influence predicted confidence. By leveraging self-context Shapley values and pairwise interactions within a greedy insertion/deletion framework, MoXI achieves a quadratic computational cost $O(|N|^2)$, avoiding the exponential burden of naive Shapley computation. Extensive experiments on ImageNet with ViT/DeiT/ResNet-18 show MoXI produces sharper insertion/deletion curves, more concise heatmaps, and improved class-discriminative localization compared with Grad-CAM, Attention rollout, and standard Shapley methods. The results demonstrate the method’s practical value for robust, interpretable visual explanations, with the code available at the project repository.

Abstract

To better understand the behavior of image classifiers, it is useful to visualize the contribution of individual pixels to the model prediction. In this study, we propose a method, MoXI ($\textbf{Mo}$del e$\textbf{X}$planation by $\textbf{I}$nteractions), that efficiently and accurately identifies a group of pixels with high prediction confidence. The proposed method employs game-theoretic concepts, Shapley values and interactions, taking into account the effects of individual pixels and the cooperative influence of pixels on model confidence. Theoretical analysis and experiments demonstrate that our method better identifies the pixels that are highly contributing to the model outputs than widely-used visualization by Grad-CAM, Attention rollout, and Shapley value. While prior studies have suffered from the exponential computational cost in the computation of Shapley value and interactions, we show that this can be reduced to quadratic cost for our task. The code is available at https://github.com/KosukeSumiyasu/MoXI.

Figures (19)

• Figure 1: Examples of image patches with high contributions to the output of ViT-T. (a) Starting from an empty image, image patches are inserted according to their contribution measured by each method. (b) Starting from an original image, image patches are removed according to their contribution measured by each method. The heatmaps highlight the image patches inserted (deleted) to obtain the correct (incorrect) classification. The selected patches are colored according to the timing of insertion/deletion. For insertion, only the proposed method selects patches from the background. For deletion, the proposed method highlights the class object only. For both cases, the proposed method highlights the least number of patches while achieving the highest/lowest confidence score.
• Figure 2: (a) Example of a synthetic MNIST image in the visual Set-Sum task, labeled 17 by the sum of all types of numbers in the image. (b) Insertion curves. The curves illustrate the change of accuracy when adding image patches gradually with high contributions identified by different methods at various unmasked image rates, ranging from $0$ to $100\%$. These curves use a masking method that fills in zeros for game-theoretic calculations and model input during classification accuracy measurement. MoXI(-) only employs self-context Shapley values, whereas MoXI additionally uses interactions across highly contributing patches.
• Figure 3: (a) Insertion curves. (b) Deletion curves. The curves illustrate the change of accuracy when appending (removing) image patches gradually with high contributions identified by different methods at various unmasked (masked) image rates, ranging from $0$ to $100\%$.
• Figure 4: Visualization of important image patches by each method. The highlighted image patches are selected based on their contributions calculated by each method. (a) Highlighting the patches incrementally added to an entire image until classification success. (b) Highlighting the patches sequentially removed from a full image until classification failure.
• Figure 5: Visualization of important region for a targeted class using the proposed method. (a) Original image. (b) Targeting the bull mastiff class, which is predicted by the model. The highlighted patches are those sequentially removed from a full image until predict the bull mastiff class. (c) Targeting tiger cat class. We first removed the patches that has a positive contribution to bull mastiff class and also negative contribution to tiger cat. Once the tiger cat becomes the predicted class of the model, the patches highly contributing to tiger cat is removed sequentially until the prediction change, which are the highlighted patches.