There is More to Attention: Statistical Filtering Enhances Explanations in Vision Transformers

TL;DR

This work tackles the gap in explainability for Vision Transformers by reframing attention as a signal rather than a nuisance. It introduces RFEM, a three-stage statistical filtering pipeline that preserves per-head attention signals across layers, applies adaptive filtering, and fuses them into faithful explanations; RFEM-Class further incorporates target-class gradients to produce discriminative, class-specific maps. Across MexCulture, SALICON, and ImageNet1K perturbations, RFEM and RFEM-Class achieve strong plausibility and competitive faithfulness, often outperforming state-of-the-art baselines while maintaining efficiency. The findings suggest that properly filtered attention can serve as a fast, human-aligned explanation modality with potential applicability to other data domains and modalities.

Abstract

Explainable AI (XAI) has become increasingly important with the rise of large transformer models, yet many explanation methods designed for CNNs transfer poorly to Vision Transformers (ViTs). Existing ViT explanations often rely on attention weights, which tend to yield noisy maps as they capture token-to-token interactions within each layer.While attribution methods incorporating MLP blocks have been proposed, we argue that attention remains a valuable and interpretable signal when properly filtered. We propose a method that combines attention maps with a statistical filtering, initially proposed for CNNs, to remove noisy or uninformative patterns and produce more faithful explanations. We further extend our approach with a class-specific variant that yields discriminative explanations. Evaluation against popular state-of-the-art methods demonstrates that our approach produces sharper and more interpretable maps. In addition to perturbation-based faithfulness metrics, we incorporate human gaze data to assess alignment with human perception, arguing that human interpretability remains essential for XAI. Across multiple datasets, our approach consistently outperforms or is comparable to the SOTA methods while remaining efficient and human plausible.

Figures (8)

• Figure 1: Explanation maps of (a) SOTA AttnLRP (b) Gradient $\times$ Relevance, (d) GradCAM, (e) Input $\times$ Gradient, (f)Attention Rollout, (g) SAW and our methods (h) RFEM and the class-specific (i) RFEM-Class
• Figure 2: Pipeline for our RFEM method. Dimensions correspond to a ViT-B16 model with 12 layers and 12 heads per layer. The yellow row highlights the query associated with the [CLS] token, used to generate the final explanation
• Figure 3: Illustration for RFEM-Class. Gradients with respect to the target class $c$, where $c \in 1, ...,C$, are computed by a backward pass and are used to weight the attention maps at each layer. The remaining steps follow the RFEM method.
• Figure 4: For the image (a) our methods (c) and (d) are better than the Attention Rollout (b) map. RFEM filters out the "unimportant" attentions to focus more on the the main objects and by including the gradient information for RFEM-Class it focuses on the main object in the image.
• Figure 5: Qualitative comparison of the XAI methods across the three classes for MexCulture images