Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LIMEtree: Consistent and Faithful Surrogate Explanations of Multiple Classes

Kacper Sokol, Peter Flach

TL;DR

This work tackles the challenge of explaining predictions across multiple classes by introducing LIMEtree, a local surrogate that uses a single multi-output regression tree to model all explained classes simultaneously. By employing a deterministic interpretable representation and a fidelity-lean objective, LIMEtree delivers model-driven and data-driven explanations with strong fidelity guarantees while offering a diverse set of explanation types, including counterfactuals, explanations of tree structure, and what-if analyses. The authors demonstrate that multi-class explanations can be more coherent and informative than independent per-class explainers, validated through quantitative fidelity tests on image and tabular data and a pilot user study. Overall, the approach provides a practical, post-hoc, model-agnostic framework for consistent and faithful multi-class explanations with potential for broad impact across XAI applications.

Abstract

Explainable artificial intelligence provides tools to better understand predictive models and their decisions, but many such methods are limited to producing insights with respect to a single class. When generating explanations for several classes, reasoning over them to obtain a comprehensive view may be difficult since they can present competing or contradictory evidence. To address this challenge we introduce the novel paradigm of multi-class explanations. We outline the theory behind such techniques and propose a local surrogate model based on multi-output regression trees -- called LIMEtree -- that offers faithful and consistent explanations of multiple classes for individual predictions while being post-hoc, model-agnostic and data-universal. On top of strong fidelity guarantees, our implementation delivers a range of diverse explanation types, including counterfactual statements favoured in the literature. We evaluate our algorithm with respect to explainability desiderata, through quantitative experiments and via a pilot user study, on image and tabular data classification tasks, comparing it to LIME, which is a state-of-the-art surrogate explainer. Our contributions demonstrate the benefits of multi-class explanations and wide-ranging advantages of our method across a diverse set of scenarios.

LIMEtree: Consistent and Faithful Surrogate Explanations of Multiple Classes

TL;DR

This work tackles the challenge of explaining predictions across multiple classes by introducing LIMEtree, a local surrogate that uses a single multi-output regression tree to model all explained classes simultaneously. By employing a deterministic interpretable representation and a fidelity-lean objective, LIMEtree delivers model-driven and data-driven explanations with strong fidelity guarantees while offering a diverse set of explanation types, including counterfactuals, explanations of tree structure, and what-if analyses. The authors demonstrate that multi-class explanations can be more coherent and informative than independent per-class explainers, validated through quantitative fidelity tests on image and tabular data and a pilot user study. Overall, the approach provides a practical, post-hoc, model-agnostic framework for consistent and faithful multi-class explanations with potential for broad impact across XAI applications.

Abstract

Explainable artificial intelligence provides tools to better understand predictive models and their decisions, but many such methods are limited to producing insights with respect to a single class. When generating explanations for several classes, reasoning over them to obtain a comprehensive view may be difficult since they can present competing or contradictory evidence. To address this challenge we introduce the novel paradigm of multi-class explanations. We outline the theory behind such techniques and propose a local surrogate model based on multi-output regression trees -- called LIMEtree -- that offers faithful and consistent explanations of multiple classes for individual predictions while being post-hoc, model-agnostic and data-universal. On top of strong fidelity guarantees, our implementation delivers a range of diverse explanation types, including counterfactual statements favoured in the literature. We evaluate our algorithm with respect to explainability desiderata, through quantitative experiments and via a pilot user study, on image and tabular data classification tasks, comparing it to LIME, which is a state-of-the-art surrogate explainer. Our contributions demonstrate the benefits of multi-class explanations and wide-ranging advantages of our method across a diverse set of scenarios.

Paper Structure

This paper contains 14 sections, 2 theorems, 7 equations, 14 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work and Background
  3. LIMEtree
  4. Fidelity Guarantees
  5. Qualitative, Quantitative and User-based Evaluation
  6. Desiderata
  7. Synthetic Experiments
  8. Pilot User Study
  9. Discussion
  10. Conclusion and Future Work
  11. LIMEtree Algorithms
  12. Proofs
  13. Loss Behaviour
  14. Examples of Diverse Explanation Types

Key Result

Lemma 1

A surrogate tree can achieve full fidelity with respect to the explanations derived from its structure -- i.e., model-driven explanations -- if the interpretable representation transformation function $\mathit{IR}$ is deterministic. Therefore, an instance $x \in \mathcal{X}$ can be translated into a

Figures (14)

  • Figure S1: LIME explanations for the top three classes predicted by a black-box model. Panel \ref{['fig:lime:image']} shows the super-pixel interpretable representation of the explained image with $d=8$ segments. Panels \ref{['fig:lime:ball']}, \ref{['fig:lime:labrador']} and \ref{['fig:lime:golden']} are LIME explanations; they capture the positive or negative influence of (the presence of) interpretable features on the prediction (probability) of a selected class.
  • Figure S2: Surrogate multi-output binary regression tree explaining the top three classes -- tennis ball, golden retriever and Labrador retriever -- predicted by a black box for the image shown in Figure \ref{['fig:lime:image']}. The segments marked in blue do not influence the explanation at a given tree node, i.e., they can either be preserved or discarded for the explanation to hold. Super-pixels whose value in the interpretable representation is $1$ are preserved and those with $0$ are "removed" by occluding them with black patches. The class probabilities estimated by each node of the surrogate tree may not sum up to $1$ as these values capture a subset of the modelled classes and are a result of numerical regression, hence they should not be treated as probabilities per se.
  • Figure S3: High-level overview of LIMEtree.
  • Figure S4: Behaviour of the LIMEtree loss (fidelity $\mathcal{L}$ and its standard deviation, y-axis) computed for the top three classes of the \ref{['fig:loss:cifar100']} CIFAR-100 and \ref{['fig:loss:forest']} Forest Covertypes data sets and plotted against surrogate complexity ($\Omega$, x-axis) given as the ratio between the depth of the tree and its maximum depth (complete tree) determined by the number of features of the interpretable domain. We report results for three surrogate variants: LIME, TREE and TREE; the plots are representative of the other data sets used in our experiments (see Appendix \ref{['apx:loss']} for the remaining figures) and complement the fidelity at fixed tree complexity levels (66%, 75% and 100%) reported in Table \ref{['tab:fidelity']}. LIME complexity is constant and given by the number of features in the interpretable representation, i.e., 100% equivalent.
  • Figure S5: High-level overview of the user study flow.
  • ...and 9 more figures

Theorems & Definitions (3)

  • Definition 1: Minimal Representation
  • Lemma 1: Structural Fidelity
  • Corollary 1: Full Fidelity