Gaussian Smoothing in Saliency Maps: The Stability-Fidelity Trade-Off in Neural Network Interpretability

TL;DR

Gradient-based saliency maps often vary with stochastic training, hindering reliable interpretation. The authors extend an algorithmic stability framework to quantify how Gaussian smoothing in Smooth-Grad affects stability and fidelity, demonstrating that stability improves roughly by a factor of $1/\sigma$ while fidelity deteriorates roughly proportionally to $\sigma$. They derive explicit bounds for Simple-Grad, Integrated-Gradients, and Smooth-Grad, and corroborate these results with CIFAR-10 and ImageNet experiments across multiple architectures. The work provides principled guidance on selecting the smoothing parameter $\sigma$ to balance stability and fidelity for different interpretability tasks. This contributes to more reliable gradient-based explanations and informs practical use of smoothing in neural network interpretability.

Abstract

Saliency maps have been widely used to interpret the decisions of neural network classifiers and discover phenomena from their learned functions. However, standard gradient-based maps are frequently observed to be highly sensitive to the randomness of training data and the stochasticity in the training process. In this work, we study the role of Gaussian smoothing in the well-known Smooth-Grad algorithm in the stability of the gradient-based maps to the randomness of training samples. We extend the algorithmic stability framework to gradient-based interpretation maps and prove bounds on the stability error of standard Simple-Grad, Integrated-Gradients, and Smooth-Grad saliency maps. Our theoretical results suggest the role of Gaussian smoothing in boosting the stability of gradient-based maps to the randomness of training settings. On the other hand, we analyze the faithfulness of the Smooth-Grad maps to the original Simple-Grad and show the lower fidelity under a more intense Gaussian smoothing. We support our theoretical results by performing several numerical experiments on standard image datasets. Our empirical results confirm our hypothesis on the fidelity-stability trade-off in the application of Gaussian smoothing to gradient-based interpretation maps.

Figures (17)

• Figure 1: The stability-fidelity trade-off introduced by Gaussian smoothing. We trained neural networks on two disjoint training set splits of ImageNet, and computed the Simple-Grad and Smooth-Grad maps for the same test sample.
• Figure 2: Relationship between stability error of saliency maps and sigma, evaluated on the CIFAR10 test set. The shaded region indicates one standard deviation from the mean value. Here noise level $x_{\text{max}}-x_{\text{min}}$ represents the largest value range for all pixels, following the practice of smilkov2017smoothgrad.
• Figure 3: Relationship between stability error of saliency maps and sigma, evaluated on the ImageNet test set. The shaded region indicates one standard deviation from the mean value.
• Figure 4: The relation between the saliency map fidelity error and sigma. This experiment is conducted on the test set of CIFAR10. The shaded area represents one standard deviation.
• Figure 5: The relation between the saliency map fidelity error and sigma. This experiment is conducted on the test set of ImageNet. The shaded area represents one standard deviation.

Theorems & Definitions (43)

• Proposition 1
• Theorem 2
• proof
• Remark 4
• Theorem 5
• proof
• Theorem 6
• proof
• Proposition 7
• proof