Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

UbiQTree: Uncertainty Quantification in XAI with Tree Ensembles

Akshat Dubey, Aleksandar Anžel, Bahar İlgen, Georges Hattab

TL;DR

This work proposes an approach for decomposing uncertainty in SHAP values into aleatoric, epistemic, and entanglement components and validate the method across three real-world use cases with descriptive statistical analyses that provide insight into the nature of epistemic uncertainty embedded in SHAP explanations.

Abstract

Explainable Artificial Intelligence (XAI) techniques, such as SHapley Additive exPlanations (SHAP), have become essential tools for interpreting complex ensemble tree-based models, especially in high-stakes domains such as healthcare analytics. However, SHAP values are usually treated as point estimates, which disregards the inherent and ubiquitous uncertainty in predictive models and data. This uncertainty has two primary sources: aleatoric and epistemic. The aleatoric uncertainty, which reflects the irreducible noise in the data. The epistemic uncertainty, which arises from a lack of data. In this work, we propose an approach for decomposing uncertainty in SHAP values into aleatoric, epistemic, and entanglement components. This approach integrates Dempster-Shafer evidence theory and hypothesis sampling via Dirichlet processes over tree ensembles. We validate the method across three real-world use cases with descriptive statistical analyses that provide insight into the nature of epistemic uncertainty embedded in SHAP explanations. The experimentations enable to provide more comprehensive understanding of the reliability and interpretability of SHAP-based attributions. This understanding can guide the development of robust decision-making processes and the refinement of models in high-stakes applications. Through our experiments with multiple datasets, we concluded that features with the highest SHAP values are not necessarily the most stable. This epistemic uncertainty can be reduced through better, more representative data and following appropriate or case-desired model development techniques. Tree-based models, especially bagging, facilitate the effective quantification of epistemic uncertainty.

UbiQTree: Uncertainty Quantification in XAI with Tree Ensembles

TL;DR

This work proposes an approach for decomposing uncertainty in SHAP values into aleatoric, epistemic, and entanglement components and validate the method across three real-world use cases with descriptive statistical analyses that provide insight into the nature of epistemic uncertainty embedded in SHAP explanations.

Abstract

Explainable Artificial Intelligence (XAI) techniques, such as SHapley Additive exPlanations (SHAP), have become essential tools for interpreting complex ensemble tree-based models, especially in high-stakes domains such as healthcare analytics. However, SHAP values are usually treated as point estimates, which disregards the inherent and ubiquitous uncertainty in predictive models and data. This uncertainty has two primary sources: aleatoric and epistemic. The aleatoric uncertainty, which reflects the irreducible noise in the data. The epistemic uncertainty, which arises from a lack of data. In this work, we propose an approach for decomposing uncertainty in SHAP values into aleatoric, epistemic, and entanglement components. This approach integrates Dempster-Shafer evidence theory and hypothesis sampling via Dirichlet processes over tree ensembles. We validate the method across three real-world use cases with descriptive statistical analyses that provide insight into the nature of epistemic uncertainty embedded in SHAP explanations. The experimentations enable to provide more comprehensive understanding of the reliability and interpretability of SHAP-based attributions. This understanding can guide the development of robust decision-making processes and the refinement of models in high-stakes applications. Through our experiments with multiple datasets, we concluded that features with the highest SHAP values are not necessarily the most stable. This epistemic uncertainty can be reduced through better, more representative data and following appropriate or case-desired model development techniques. Tree-based models, especially bagging, facilitate the effective quantification of epistemic uncertainty.

Paper Structure

This paper contains 13 sections, 5 theorems, 34 equations, 15 figures, 5 algorithms.

Table of Contents

  1. Background and Approaches to Uncertainty Quantification in XAI:
  2. SHAP Specific Uncertainty Quantification:
  3. Information Theory Approaches and Other Alternatives:
  4. Step 1: Total Variance Definition
  5. Step 2: Expand $\mathbb{E}_{D,f}[\phi_i^2]$ using Law of Total Expectation
  6. Step 3: Expand $\left(\mathbb{E}_{D,f}[\phi_i]\right)^2$
  7. Step 4: Combine Terms
  8. Step 5: Identify the Entanglement Term:
  9. Step 6: Final Decomposition:
  10. Key Insights:
  11. For the reproducibility of these experiments, the following experimental details are provided:
  12. For the reproducibility of these experiments, the following experimental details are provided:
  13. For the reproducibility of these experiments, the following experimental details are provided:

Key Result

Theorem 1

For any tree ensemble model $f$, the point estimate SHAP value $\phi_i$ lacks a measure of variance V($\phi_i$|f, D) over possible training datasets $D\sim P_\text{data}$. This violates the reliability axiom for explainability in high-risk AI systems balagurunathan2021requirements.

Figures (15)

  • Figure 1:
  • Figure 2:
  • Figure 3:
  • Figure 4:
  • Figure 5:
  • ...and 10 more figures

Theorems & Definitions (5)

  • Theorem 1
  • Theorem 2: Uncertainty Distribution
  • Lemma 1: Optimal Acquisition
  • Theorem 3: Constructing the Dirichlet Process
  • Theorem 4: Convergence