Unifying Perplexing Behaviors in Modified BP Attributions through Alignment Perspective

TL;DR

This work provides a unified theoretical framework for modified backpropagation attribution methods by framing them as input-alignment processes that cascade through activated neurons via a Negative Filtering Rule (NFR). It unifies GBP, RectGrad, LRP, and DTD under the same mechanism, explains their interpretability and insensitivity to weight randomization, and proves two theorems describing input alignment under isotropic randomness and cascade preservation under orthogonal, equal-norm weights. The authors validate the theory with extensive experiments on ImageNet and CIFAR-10, introduce the Key Information Sufficiency (KIS) metric to quantify how attribution-identified input information drives decisions, and demonstrate the framework’s utility in failure analysis and backdoor evaluation. Overall, the results offer actionable insights into how attributions reflect decision-relevant input information, how this information evolves across layers, and how to assess attribution reliability beyond traditional sanity checks.

Abstract

Attributions aim to identify input pixels that are relevant to the decision-making process. A popular approach involves using modified backpropagation (BP) rules to reverse decisions, which improves interpretability compared to the original gradients. However, these methods lack a solid theoretical foundation and exhibit perplexing behaviors, such as reduced sensitivity to parameter randomization, raising concerns about their reliability and highlighting the need for theoretical justification. In this work, we present a unified theoretical framework for methods like GBP, RectGrad, LRP, and DTD, demonstrating that they achieve input alignment by combining the weights of activated neurons. This alignment improves the visualization quality and reduces sensitivity to weight randomization. Our contributions include: (1) Providing a unified explanation for multiple behaviors, rather than focusing on just one. (2) Accurately predicting novel behaviors. (3) Offering insights into decision-making processes, including layer-wise information changes and the relationship between attributions and model decisions.

Figures (9)

• Figure 1: The geometric intuition of the proof of Theorem \ref{['theorem1']}. Masking negative vectors can improve input alignment.
• Figure 2: (a) Cascadingly replacing the raw gradient with BP rules from GBP, RectGrad, and the $z^+$ rule, along with activations from the top layer to the bottom layer, results in a consistent increase in the cosine similarity between the final attribution and the input. (b) For each layer, the cosine similarity between the weights of corresponding neurons is computed, where a similarity of 0 indicates orthogonality. (c) For each layer, the $L_2$ norm of each neuron's weight is computed, with smaller error bars indicating lower standard deviation in the $L_2$ norm of the layer weights.
• Figure 3: Left: Illustration of weight randomization distributions: the top shows a Gaussian distribution, the middle a non-Gaussian but isotropic distribution, and the bottom a non-isotropic distribution. Our theory predicts that the bottom distribution deviates from the input $\textbf{x}$. Right: All randomization methods show alignment with the input image. However, the lack of isotropy in the weight distribution causes a more significant distortion in the color results of uniform randomization compared to Gaussian or ring randomization.
• Figure 4: The quantitative results of cascade substitution for different distributions show that ring randomization provides similar alignment performance to Gaussian randomization, while uniform randomization performs significantly worse across all NFR methods.
• Figure 5: The removal operation results in a significant loss of content in the interpreted results, suggesting that these methods are not inherently insensitive to weight changes in the top layers.

Theorems & Definitions (8)

• Definition 1
• Definition 2
• Proposition 1
• Proposition 2
• Proposition 3
• Theorem 1
• Theorem 2
• Definition 3