Statistics and explainability: a fruitful alliance

TL;DR

The paper argues that many core explainability challenges stem from vague definitions, lack of guarantees, and poor evaluation. It proposes a statistics-based framework that defines explanations as statistical quantities (e.g., variable importance), establishes convergence guarantees, and enables objective evaluation and uncertainty quantification via methods like the bootstrap. Key contributions include a formal definition of explanations, convergence results, and a blueprint for confidence assessment, along with demonstrations of how classical statistical tools can yield trustworthy, interpretable, and fair explanations. The authors acknowledge limits of statistics alone and outline open problems such as defining purpose, ensuring simplicity, and extending the framework to counterfactual and adversarial settings, aiming to provide a principled foundation for explainable AI in practice.

Abstract

In this paper, we propose standard statistical tools as a solution to commonly highlighted problems in the explainability literature. Indeed, leveraging statistical estimators allows for a proper definition of explanations, enabling theoretical guarantees and the formulation of evaluation metrics to quantitatively assess the quality of explanations. This approach circumvents, among other things, the subjective human assessment currently prevalent in the literature. Moreover, we argue that uncertainty quantification is essential for providing robust and trustworthy explanations, and it can be achieved in this framework through classical statistical procedures such as the bootstrap. However, it is crucial to note that while Statistics offers valuable contributions, it is not a panacea for resolving all the challenges. Future research avenues could focus on open problems, such as defining a purpose for the explanations or establishing a statistical framework for counterfactual or adversarial scenarios.