Improving Interpretation Faithfulness for Vision Transformers

TL;DR

This work addresses the fragility of attention-based explanations in Vision Transformers by introducing Faithful Vision Transformers (FViTs) and a constructive method called Denoised Diffusion Smoothing (DDS). DDS combines randomized smoothing with diffusion-based denoising to produce attention and predictions that remain stable under input perturbations, effectively turning ViTs into FViTs. The authors provide a formal definition of faithfulness based on stable top-$k$ attention and robust prediction distributions, prove conditions under which DDS yields a faithful attention module, and show near-optimality of Gaussian noise for both $\ell_2$ and $\ell_\infty$ norms. Extensive experiments on ImageNet, Cityscape, and COCO across ViT, DeiT, and Swin models demonstrate improved interpretability faithfulness and robustness to adversarial perturbations, with a favorable computation cost, suggesting practical applicability to larger LVLMs and safety-focused AI systems.

Abstract

Vision Transformers (ViTs) have achieved state-of-the-art performance for various vision tasks. One reason behind the success lies in their ability to provide plausible innate explanations for the behavior of neural architectures. However, ViTs suffer from issues with explanation faithfulness, as their focal points are fragile to adversarial attacks and can be easily changed with even slight perturbations on the input image. In this paper, we propose a rigorous approach to mitigate these issues by introducing Faithful ViTs (FViTs). Briefly speaking, an FViT should have the following two properties: (1) The top-$k$ indices of its self-attention vector should remain mostly unchanged under input perturbation, indicating stable explanations; (2) The prediction distribution should be robust to perturbations. To achieve this, we propose a new method called Denoised Diffusion Smoothing (DDS), which adopts randomized smoothing and diffusion-based denoising. We theoretically prove that processing ViTs directly with DDS can turn them into FViTs. We also show that Gaussian noise is nearly optimal for both $\ell_2$ and $\ell_\infty$-norm cases. Finally, we demonstrate the effectiveness of our approach through comprehensive experiments and evaluations. Results show that FViTs are more robust against adversarial attacks while maintaining the explainability of attention, indicating higher faithfulness.

Figures (10)

• Figure 1: Visualization results of the attention map on corrupted input for different methods.
• Figure 2: Class-specific explanation heat map visualizations under adversarial corruption. For each image, we present results for two different classes. Other baselines either give inconsistent interpretations or show wrong focus class regions under adversarial perturbations. While our method gives a consistent interpretation map and is robust against adversarial attacks.
• Figure 3: (a) and (b) are results of the perturbation test. (c) is the sensitivity analysis results.
• Figure 4: Class-specific explanation heat map visualizations under adversarial corruption. For each image, we present results for two different classes. Other baselines either give inconsistent interpretations or show wrong focus class regions under adversarial perturbations. While our method gives a consistent interpretation map and is robust against adversarial attacks.
• Figure 5: Visualization results of the attention map on corrupted input for different methods.

Theorems & Definitions (26)

• Definition 4.1
• Definition 4.2: Faithful ViTs
• Definition 4.3
• Theorem 4.4
• Theorem 5.1
• Theorem 5.2
• Theorem 5.3
• Theorem 5.4
• proof
• Lemma 2.1: Rényi Divergence Lemma li2019certified