Uncertainty-Aware Subset Selection for Robust Visual Explainability under Distribution Shifts

TL;DR

The paper addresses the robustness of subset-based visual explanations under distribution shifts by introducing an uncertainty-aware framework that integrates gradient-based uncertainty with submodular subset selection. It develops two objective functions for visual attribution and object-level interpretation, both leveraging a gradient-derived confidence score computed via adaptive weight perturbations and layer-wise gradients, without requiring training-time changes. The approach yields consistent improvements in attribution fidelity across ID and OOD settings and demonstrates strong robustness to various distribution shifts, supported by both quantitative metrics and qualitative results. This work advances transparent AI in vision by making explanations more reliable, stable, and semantically meaningful under real-world data distributions.

Abstract

Subset selection-based methods are widely used to explain deep vision models: they attribute predictions by highlighting the most influential image regions and support object-level explanations. While these methods perform well in in-distribution (ID) settings, their behavior under out-of-distribution (OOD) conditions remains poorly understood. Through extensive experiments across multiple ID-OOD sets, we find that reliability of the existing subset based methods degrades markedly, yielding redundant, unstable, and uncertainty-sensitive explanations. To address these shortcomings, we introduce a framework that combines submodular subset selection with layer-wise, gradient-based uncertainty estimation to improve robustness and fidelity without requiring additional training or auxiliary models. Our approach estimates uncertainty via adaptive weight perturbations and uses these estimates to guide submodular optimization, ensuring diverse and informative subset selection. Empirical evaluations show that, beyond mitigating the weaknesses of existing methods under OOD scenarios, our framework also yields improvements in ID settings. These findings highlight limitations of current subset-based approaches and demonstrate how uncertainty-driven optimization can enhance attribution and object-level interpretability, paving the way for more transparent and trustworthy AI in real-world vision applications.

Figures (5)

• Figure 1: Attribution map for an ID sample: Semantically coherent regions highlight the bird's key features
• Figure 2: Attribution map for an OOD sample: Quality of selected subsets degrades as they become fragmented and show irrelevant background
• Figure 3: An overview of the uncertainty-aware submodular selection framework. The process evaluates candidate image patches, generated by a baseline method, to select the most informative subset for an explanation. This selection is guided by a submodular objective function where all 4 component scores-Confidence, Effectiveness, Consistency and Collaboration, are derived from a single Recognition Model. Our novel Confidence Score is calculated by measuring the model's output stability under adaptive noise perturbations. The framework greedily selects a compact and reliable explanation, with its final quality assessed by metrics like the Insertion AUC score (right).
• Figure 4: Qualitative comparison on OOD samples: The figure contrasts the baseline HSIC+SMDL (left) with our proposed method (right) on a 'sweet pepper' from CIFAR-100 (top) and a bird from North American Birds (bottom).
• Figure 5: Qualitative comparison for object-level interpretation on OOD examples: This figure contrasts the baseline VPS method (left) with our proposed method (right) on a transformed COCO image (top) and an iNaturalist image (bottom)