Evaluating Feature Attribution Methods in the Image Domain

TL;DR

This work tackles the objective evaluation of image feature attribution maps by surveying existing metrics, introducing three novel metrics, and providing a comprehensive benchmarking pipeline across 8 datasets and 14 attribution methods. It demonstrates that attribution metrics probe different underlying properties and that results strongly depend on the dataset, challenging the notion of a universal best method. The study finds complementarity between coarse-grained and fine-grained attribution approaches and shows that theoretically grounded methods like DeepSHAP do not universally outperform cheaper alternatives. By offering a practical benchmarking guideline, the paper enables use-case–driven selection of attribution methods, balancing accuracy, computational cost, and desired granularity. The work thus advances reliable interpretation practices for deep image models and highlights avenues for dataset-specific evaluation in real-world deployments.

Abstract

Feature attribution maps are a popular approach to highlight the most important pixels in an image for a given prediction of a model. Despite a recent growth in popularity and available methods, little attention is given to the objective evaluation of such attribution maps. Building on previous work in this domain, we investigate existing metrics and propose new variants of metrics for the evaluation of attribution maps. We confirm a recent finding that different attribution metrics seem to measure different underlying concepts of attribution maps, and extend this finding to a larger selection of attribution metrics. We also find that metric results on one dataset do not necessarily generalize to other datasets, and methods with desirable theoretical properties such as DeepSHAP do not necessarily outperform computationally cheaper alternatives. Based on these findings, we propose a general benchmarking approach to identify the ideal feature attribution method for a given use case. Implementations of attribution metrics and our experiments are available online.

Figures (26)

• Figure 1: Results of paired t-tests (low-dimensional datasets). A square is only drawn if the corresponding result was significant after Bonferroni correction ($p < 0.01$).
• Figure 2: Results of paired t-tests (medium-dimensional datasets). A square is only drawn if the corresponding result was significant after Bonferroni correction ($p < 0.01$).
• Figure 3: Results of paired t-tests (high-dimensional datasets). A square is only drawn if the corresponding result was significant after Bonferroni correction ($p < 0.01$).
• Figure 4: Average inter-metric correlations for low-, medium- and high-dimensional datasets. Impact Coverage (Cov) was only computed for the high-dimensional datasets due to the requirement of an adversarial patch (see Section \ref{['sec:impact-coverage']})
• Figure 5: Krippendorff's $\alpha$ for default implementations of different metrics on all datasets. Low-, medium- and high-dimensional datasets are indicated in green, blue and red tones, respectively. Impact Coverage (Cov) was only computed for the high-dimensional datasets due to the requirement of an adversarial patch (see Section \ref{['sec:impact-coverage']})