Better Understanding Differences in Attribution Methods via Systematic Evaluations

TL;DR

This work tackles the challenge of fairly evaluating post-hoc attribution methods for deep networks by proposing three complementary schemes: DiFull controls which input parts can influence the output; DiPart relaxes full disconnection to reflect realistic receptive-field effects; and AggAtt provides systematic qualitative visualizations across complete datasets. By applying these schemes to seven models and multiple attribution methods across ImageNet and CIFAR-10, the authors reveal that several methods perform similarly when comparisons are fair, and show how smoothing (S-IntGrad, S-IxG) can substantially improve localization. They also analyze correlations across methods and layers, discuss computational costs, and expose invariances (or lack thereof) in LRP. The findings offer practical guidance for interpreting attribution results and for choosing evaluation setups that yield reliable, comparable insights into method faithfulness and locality.

Abstract

Deep neural networks are very successful on many vision tasks, but hard to interpret due to their black box nature. To overcome this, various post-hoc attribution methods have been proposed to identify image regions most influential to the models' decisions. Evaluating such methods is challenging since no ground truth attributions exist. We thus propose three novel evaluation schemes to more reliably measure the faithfulness of those methods, to make comparisons between them more fair, and to make visual inspection more systematic. To address faithfulness, we propose a novel evaluation setting (DiFull) in which we carefully control which parts of the input can influence the output in order to distinguish possible from impossible attributions. To address fairness, we note that different methods are applied at different layers, which skews any comparison, and so evaluate all methods on the same layers (ML-Att) and discuss how this impacts their performance on quantitative metrics. For more systematic visualizations, we propose a scheme (AggAtt) to qualitatively evaluate the methods on complete datasets. We use these evaluation schemes to study strengths and shortcomings of some widely used attribution methods over a wide range of models. Finally, we propose a post-processing smoothing step that significantly improves the performance of some attribution methods, and discuss its applicability.

Figures (28)

• Figure 1: Spearman rank correlation coefficients between Grad-CAM at the final layer and S-IntGrad at the input layer on GridPG for varying degrees of smoothing. The first column shows the correlation with the original unsmoothed version. We observe that the correlation improves significantly for both VGG19 and Resnet152 when smoothing with large kernel sizes.
• Figure 2: Spearman rank correlation coefficients between Grad-CAM at the final layer and S-IxG at the input layer on GridPG for varying degrees of smoothing. The first column shows the correlation with the original unsmoothed version. We observe that the correlation improves for VGG19, but does not significantly improve for Resnet152.
• Figure 11: Quantitative Results on ImageNet. We evaluate the localization scores each attribution method at the input (Inp), middle (Mid), and final (Fin) convolutional layers, on each of GridPG, DiFull, and DiPart using seven models. Top: Backpropagation-based methods. Middle: Activation-based methods. Bottom: Perturbation-based methods. The two horizontal dotted lines mark localization scores of $1.0$ and $0.25$, which correspond to perfect and random localization, respectively.
• Figure 12: Examples from each AggAtt bin for each method at the input layer on GridPG using VGG19. From each bin, the image and its attribution at the median position are shown.
• Figure 13: Examples from each AggAtt bin for each method at the final layer on GridPG using VGG19. From each bin, the image and its attribution at the median position are shown.