Estimation of Concept Explanations Should be Uncertainty Aware
Vihari Piratla, Juyeon Heo, Katherine M. Collins, Sukriti Singh, Adrian Weller
TL;DR
This work tackles the noise and data inefficiency of concept-based explanations by introducing U-ACE, a Bayesian estimator that explicitly models uncertainty in concept activations and learns a noise-robust mapping to model logits. Using a multimodal CLIP framework to define concepts from text descriptions, U-ACE derives per-example activation intervals and a posterior over explanation weights, yielding more stable, label-efficient explanations even under misspecified concept sets. Theoretical results show robustness advantages over standard estimators in over- and under-complete concept sets, while empirical evaluations across controlled and real-world datasets demonstrate improved alignment with ground-truth explanations and reduced sensitivity to probe datasets and dataset shifts. The approach demonstrates strong potential for reliable model interpretation in complex, open-world settings and is released with code for broader adoption and further validation.
Abstract
Model explanations can be valuable for interpreting and debugging predictive models. We study a specific kind called Concept Explanations, where the goal is to interpret a model using human-understandable concepts. Although popular for their easy interpretation, concept explanations are known to be noisy. We begin our work by identifying various sources of uncertainty in the estimation pipeline that lead to such noise. We then propose an uncertainty-aware Bayesian estimation method to address these issues, which readily improved the quality of explanations. We demonstrate with theoretical analysis and empirical evaluation that explanations computed by our method are robust to train-time choices while also being label-efficient. Further, our method proved capable of recovering relevant concepts amongst a bank of thousands, in an evaluation with real-datasets and off-the-shelf models, demonstrating its scalability. We believe the improved quality of uncertainty-aware concept explanations make them a strong candidate for more reliable model interpretation. We release our code at https://github.com/vps-anonconfs/uace.