Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Even-Ifs From If-Onlys: Are the Best Semi-Factual Explanations Found Using Counterfactuals As Guides?

Saugat Aryal, Mark T. Keane

TL;DR

This work performs comprehensive tests of 8 semi-factual methods on 7 datasets using 5 key metrics, to determine whether counterfactual guidance is necessary to find the best semi-factuals and suggests not, but rather that computing other aspects of the decision space lead to better semi-factual XAI.

Abstract

Recently, counterfactuals using "if-only" explanations have become very popular in eXplainable AI (XAI), as they describe which changes to feature-inputs of a black-box AI system result in changes to a (usually negative) decision-outcome. Even more recently, semi-factuals using "even-if" explanations have gained more attention. They elucidate the feature-input changes that do not change the decision-outcome of the AI system, with a potential to suggest more beneficial recourses. Some semi-factual methods use counterfactuals to the query-instance to guide semi-factual production (so-called counterfactual-guided methods), whereas others do not (so-called counterfactual-free methods). In this work, we perform comprehensive tests of 8 semi-factual methods on 7 datasets using 5 key metrics, to determine whether counterfactual guidance is necessary to find the best semi-factuals. The results of these tests suggests not, but rather that computing other aspects of the decision space lead to better semi-factual XAI.

Even-Ifs From If-Onlys: Are the Best Semi-Factual Explanations Found Using Counterfactuals As Guides?

TL;DR

This work performs comprehensive tests of 8 semi-factual methods on 7 datasets using 5 key metrics, to determine whether counterfactual guidance is necessary to find the best semi-factuals and suggests not, but rather that computing other aspects of the decision space lead to better semi-factual XAI.

Abstract

Recently, counterfactuals using "if-only" explanations have become very popular in eXplainable AI (XAI), as they describe which changes to feature-inputs of a black-box AI system result in changes to a (usually negative) decision-outcome. Even more recently, semi-factuals using "even-if" explanations have gained more attention. They elucidate the feature-input changes that do not change the decision-outcome of the AI system, with a potential to suggest more beneficial recourses. Some semi-factual methods use counterfactuals to the query-instance to guide semi-factual production (so-called counterfactual-guided methods), whereas others do not (so-called counterfactual-free methods). In this work, we perform comprehensive tests of 8 semi-factual methods on 7 datasets using 5 key metrics, to determine whether counterfactual guidance is necessary to find the best semi-factuals. The results of these tests suggests not, but rather that computing other aspects of the decision space lead to better semi-factual XAI.
Paper Structure (17 sections, 28 equations, 3 figures, 1 table)

This paper contains 17 sections, 28 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Semi-Factuals: Counterfactual-Free Or -Guided
  3. Counterfactual-Free Methods
  4. Local-Region Model.
  5. Diverse Semifactual Explanations of Reject (DSER).
  6. Most Distant Neighbor (MDN).
  7. Explanations for Positive Outcomes (S-GEN).
  8. Counterfactual-Guided Methods
  9. Knowledge-Light Explanation-Oriented Retrieval (KLEOR).
  10. PlausIble Exceptionality-based Contrastive Explanations (PIECE).
  11. Class-to-Class Variational Autoencoder (C2C-VAE).
  12. Diverse Counterfactual Explanations (DiCE).
  13. Experimental Evaluations: To Guide or Not to Guide?
  14. Method: Metrics
  15. Method: Datasets & Setup
  16. ...and 2 more sections

Figures (3)

  • Figure 1: Visualisation of (a) Counterfactual-Free and (b) Counterfactual-Guided Semi-factual Methods. The semi-factual, SF, is found by some computation (a) to be within the query-class at some distance from the query, Q, or (b) that is guided by the location of a counterfactual, CF, for the query, Q.
  • Figure 2: Mean Ranks of Counterfactual-Free (black) and Counterfactual-Guided (grey) Semi-Factual Methods
  • Figure 3: Median Ranks (across datasets) of Counterfactual-Free (left) and Counterfactual-Guided (right) methods for each measure. Points away from the center of graphs represent higher rank (better performance).