Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Ranking Counterfactual Explanations

Suryani Lim, Henri Prade, Gilles Richard

TL;DR

The paper formalizes counterfactual explanations and links them to factual explanations, then introduces a ranking framework to identify an optimal counterfactual via counterfactual power $cf_S(b,a) = |S_b ∩ B(a,b)|$ and a hyperball $B(a,b)$. It defines three quality metrics—typicality, capacity, and universality—to assess counterfactual explanations and validates the approach on 12 real-world categorical datasets, finding that a unique optimal counterfactual often exists and outperforms random minimal ones. The work is model-agnostic, provides a principled duality between factual and counterfactual explanations, and offers practical guidance for presenting robust, generalizable counterfactuals while acknowledging immutability constraints and GDPR considerations. Future work includes human-centered validation and deeper probabilistic analysis of the proposed metrics.

Abstract

AI-driven outcomes can be challenging for end-users to understand. Explanations can address two key questions: "Why this outcome?" (factual) and "Why not another?" (counterfactual). While substantial efforts have been made to formalize factual explanations, a precise and comprehensive study of counterfactual explanations is still lacking. This paper proposes a formal definition of counterfactual explanations, proving some properties they satisfy, and examining the relationship with factual explanations. Given that multiple counterfactual explanations generally exist for a specific case, we also introduce a rigorous method to rank these counterfactual explanations, going beyond a simple minimality condition, and to identify the optimal ones. Our experiments with 12 real-world datasets highlight that, in most cases, a single optimal counterfactual explanation emerges. We also demonstrate, via three metrics, that the selected optimal explanation exhibits higher representativeness and can explain a broader range of elements than a random minimal counterfactual. This result highlights the effectiveness of our approach in identifying more robust and comprehensive counterfactual explanations.

Ranking Counterfactual Explanations

TL;DR

The paper formalizes counterfactual explanations and links them to factual explanations, then introduces a ranking framework to identify an optimal counterfactual via counterfactual power and a hyperball . It defines three quality metrics—typicality, capacity, and universality—to assess counterfactual explanations and validates the approach on 12 real-world categorical datasets, finding that a unique optimal counterfactual often exists and outperforms random minimal ones. The work is model-agnostic, provides a principled duality between factual and counterfactual explanations, and offers practical guidance for presenting robust, generalizable counterfactuals while acknowledging immutability constraints and GDPR considerations. Future work includes human-centered validation and deeper probabilistic analysis of the proposed metrics.

Abstract

AI-driven outcomes can be challenging for end-users to understand. Explanations can address two key questions: "Why this outcome?" (factual) and "Why not another?" (counterfactual). While substantial efforts have been made to formalize factual explanations, a precise and comprehensive study of counterfactual explanations is still lacking. This paper proposes a formal definition of counterfactual explanations, proving some properties they satisfy, and examining the relationship with factual explanations. Given that multiple counterfactual explanations generally exist for a specific case, we also introduce a rigorous method to rank these counterfactual explanations, going beyond a simple minimality condition, and to identify the optimal ones. Our experiments with 12 real-world datasets highlight that, in most cases, a single optimal counterfactual explanation emerges. We also demonstrate, via three metrics, that the selected optimal explanation exhibits higher representativeness and can explain a broader range of elements than a random minimal counterfactual. This result highlights the effectiveness of our approach in identifying more robust and comprehensive counterfactual explanations.

Paper Structure

This paper contains 17 sections, 5 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. Basic definitions and notations
  5. Factual explanations
  6. Counterfactual explanations
  7. Properties of counterfactual explanations
  8. Ranking counterfactual explanations
  9. Minimal counterfactual explanations
  10. Optimal counterfactual explanations
  11. Typicality, capacity and universality metrics
  12. Experiments
  13. Counterfactuals versus Hamming distance
  14. Minimal counterfactuals ranking
  15. Optimal counterfactual evaluation
  16. ...and 2 more sections

Figures (6)

  • Figure 1: The number of counterfactual for Hamming distance 1 to $n$, where $n$ is the number of features
  • Figure 2: Average proportion of unique optimal per dataset
  • Figure 3: Average gap in counterfactual power
  • Figure 4: Average typicality: optimal versus random
  • Figure 5: Average capacity: optimal versus random (non binary classes)
  • ...and 1 more figures

Theorems & Definitions (7)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Definition 5
  • Definition 6
  • Definition 7