REPEAT: Improving Uncertainty Estimation in Representation Learning Explainability

TL;DR

The paper tackles the lack of a true certainty measure for pixel importance in unsupervised representation learning (R-XAI). REPEAT models each pixel as a Bernoulli random variable with probability $p_{ij}$ of being important and derives uncertainty as $U_{ij}=p_{ij}(1-p_{ij})$ by aggregating $K$ thresholded samples from a base stochastic R-XAI method. Across multiple datasets and encoders, REPEAT achieves more intuitive certainty estimates, stronger out-of-distribution detection, and lower uncertainty complexity than state-of-the-art baselines, including RELAX and Kernel SHAP. The method is flexible and can deploy with different stochastic R-XAI bases, enabling safer, more transparent representations in unsupervised settings.

Abstract

Incorporating uncertainty is crucial to provide trustworthy explanations of deep learning models. Recent works have demonstrated how uncertainty modeling can be particularly important in the unsupervised field of representation learning explainable artificial intelligence (R-XAI). Current R-XAI methods provide uncertainty by measuring variability in the importance score. However, they fail to provide meaningful estimates of whether a pixel is certainly important or not. In this work, we propose a new R-XAI method called REPEAT that addresses the key question of whether or not a pixel is \textit{certainly} important. REPEAT leverages the stochasticity of current R-XAI methods to produce multiple estimates of importance, thus considering each pixel in an image as a Bernoulli random variable that is either important or unimportant. From these Bernoulli random variables we can directly estimate the importance of a pixel and its associated certainty, thus enabling users to determine certainty in pixel importance. Our extensive evaluation shows that REPEAT gives certainty estimates that are more intuitive, better at detecting out-of-distribution data, and more concise.

Figures (5)

• Figure 1: Motivating example to show the difference between current uncertainty estimation techniques in R-XAI and our proposed approach. An image from Pascal-VOC Everingham2009 is encoded into a new representation using a ResNet18 feature extractor resnet and important pixels are determined. The importance (red indicates high and blue indicates low importance) is accompanied by uncertainties that specify how confident the importance of a pixel is. The results show that while the importance estimates somewhat agree, the uncertainty estimates are very different, which is due to the different type of uncertainty being captured (variation in pixel importance vs. certainty of importance).
• Figure 2: Overview of REPEAT. An image is transformed into a new representation and pixel importance is estimated using stochastic R-XAI. A histogram is constructed from pixel importance and Bernoulli samples are generated by thresholding the importance scores into foreground and background. Then, importance and uncertainty is estimated using the Bernoulli samples.
• Figure 3: Top row: input image from Pascal VOC Everingham2009, importance and histogram of similarities. Bottom row: thresholding of importance.
• Figure 4: Qualitative examples on images from PASCAL VOC.
• Figure 5: Histogram of aggregated uncertainty scores for in-distribution (PASCAL-VOC in green) and OOD (EuroSAT in red) data. This example illustrates how REPEAT gives the clearest separation between in-distribution and OOD data under the assumption that OOD data should have highest uncertainty.