VOICE: Variance of Induced Contrastive Explanations to quantify Uncertainty in Neural Network Interpretability

TL;DR

VOICE introduces a plug-in framework to visualize and quantify the predictive uncertainty of gradient-based visual explanations. It formalizes the uncertainty decomposition as $V[y] = V[E(y|S_x)] + E[V(y|S_x)]$ with $S_x = \mathcal{M}_m(f,x,P) \odot x$ and uses induced contrastive explanations to quantify the residual term via a pixelwise variance, producing a VOICE heatmap $u_m$. By drawing on $R$ contrastive explanations for classes above a threshold $p_t$, VOICE provides two objective metrics, IoU and SNR, to assess the overlap and dispersion between explanations and their uncertainty. Empirical results on ImageNet, CIFAR-10C, and multiple architectures show that uncertainty often overlaps with the explanation, particularly under incorrect predictions, and behaves similarly to epistemic uncertainty, offering a robust tool for evaluating gradient-based explanations. The work emphasizes the need to report uncertainty alongside explanation evaluations and suggests extending uncertainty quantification to non-gradient explanations in future work.

Abstract

In this paper, we visualize and quantify the predictive uncertainty of gradient-based post hoc visual explanations for neural networks. Predictive uncertainty refers to the variability in the network predictions under perturbations to the input. Visual post hoc explainability techniques highlight features within an image to justify a network's prediction. We theoretically show that existing evaluation strategies of visual explanatory techniques partially reduce the predictive uncertainty of neural networks. This analysis allows us to construct a plug in approach to visualize and quantify the remaining predictive uncertainty of any gradient-based explanatory technique. We show that every image, network, prediction, and explanatory technique has a unique uncertainty. The proposed uncertainty visualization and quantification yields two key observations. Firstly, oftentimes under incorrect predictions, explanatory techniques are uncertain about the same features that they are attributing the predictions to, thereby reducing the trustworthiness of the explanation. Secondly, objective metrics of an explanation's uncertainty, empirically behave similarly to epistemic uncertainty. We support these observations on two datasets, four explanatory techniques, and six neural network architectures. The code is available at https://github.com/olivesgatech/VOICE-Uncertainty.

Figures (7)

• Figure 1: GradCAM selvaraju2017grad explanations and the proposed uncertainty visualization. Top row are results obtained on VGG-16 simonyan2014very while bottom row are results obtained on Swin Transformer liu2021swin. Figs. (a), (b), (e), and (f) are obtained on a clean image where both VGG-16 and Swin Transformer predict correctly while Figs. (c), (d), (g), and (h) are results on noisy image where both networks predict incorrectly.
• Figure 2: Proposed VOICE uncertainty generation explained using three steps. A prediction $P$ is made in Step 1. Contrastive explanations are induced using $R$ contrast classes in Step 2. $R$ is the number of class probabilities that exceed a threshold $p_t$. In Step 3, variance across vertically stacked explanations is calculated pixelwise and normalized to produce VOICE uncertainty. The red boxes are plug-in frameworks. Green $p_t$ is a hyperparameter.
• Figure 3: Proposed VOICE uncertainty on GradCAM selvaraju2017grad explanation with multiple threshold probabilities $p_t$ with (a) $p_t = 0.001$, (b) $p_t = 0.0001$, (c) $p_t = 0.00001$, (d) $p_t = 0.000001$, (e) $p_t = 0.0000001$.
• Figure 4: Visualization of GradCAM, GradCAM++, Guided Backpropagation, and SmoothGrad explanations and their corresponding uncertainties on three correctly classified and two incorrectly classified randomly selected images from ImageNet validation set. Guided Backpropagation uncertainty is not easily visible and red boxes are used to show highlights.
• Figure 5: Explanations and uncertainty of a Spoonbill perturbed with AWGN at noise power levels of (a) 0, (b) 450, and (c) 11000, respectively. At 11000, the network incorrectly predicts a coral reef. (d) provides normalized log-likelihood, IoU and SNR metrics across $15$ noise levels. Shading indicates variance across three runs. Background green indicates correct prediction and red indicates incorrect prediction.