Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Weak Robust Compatibility Between Learning Algorithms and Counterfactual Explanation Generation Algorithms

Ao Xu, Tieru Wu

TL;DR

This work tackles robustness in counterfactual explanation generation by introducing Weak Robust Compatibility ($WRC$), a notion that couples Counterfactual Explanation Generation Algorithms ($CEGAs$) with Learning Algorithms ($LAs$) through explanatory strength measured by $d(x,(h,x))$. It defines $WRC$ and a practical WRC-Test to produce more robust counterfactuals, and grounds the analysis in a PAC-learning perspective by introducing PAC $WRC$-Approximability and associated oracle inequalities. Theoretical results show how bounds on excess risk translate into bounds on the $WRC$ gap under Tsybakov’s noise and smoothness assumptions, providing sufficient conditions for PAC $WRC$-Approximability. Empirical results on four binary datasets demonstrate that applying the WRC-Test improves robustness metrics (e.g., LOF, $WRC$) and validity for state-of-the-art CEGAs, supporting the practical utility of aligning explanation generation with the learning process.

Abstract

Counterfactual explanation generation is a powerful method for Explainable Artificial Intelligence. It can help users understand why machine learning models make specific decisions, and how to change those decisions. Evaluating the robustness of counterfactual explanation algorithms is therefore crucial. Previous literature has widely studied the robustness based on the perturbation of input instances. However, the robustness defined from the perspective of perturbed instances is sometimes biased, because this definition ignores the impact of learning algorithms on robustness. In this paper, we propose a more reasonable definition, Weak Robust Compatibility, based on the perspective of explanation strength. In practice, we propose WRC-Test to help us generate more robust counterfactuals. Meanwhile, we designed experiments to verify the effectiveness of WRC-Test. Theoretically, we introduce the concepts of PAC learning theory and define the concept of PAC WRC-Approximability. Based on reasonable assumptions, we establish oracle inequalities about weak robustness, which gives a sufficient condition for PAC WRC-Approximability.

Weak Robust Compatibility Between Learning Algorithms and Counterfactual Explanation Generation Algorithms

TL;DR

This work tackles robustness in counterfactual explanation generation by introducing Weak Robust Compatibility (), a notion that couples Counterfactual Explanation Generation Algorithms () with Learning Algorithms () through explanatory strength measured by . It defines and a practical WRC-Test to produce more robust counterfactuals, and grounds the analysis in a PAC-learning perspective by introducing PAC -Approximability and associated oracle inequalities. Theoretical results show how bounds on excess risk translate into bounds on the gap under Tsybakov’s noise and smoothness assumptions, providing sufficient conditions for PAC -Approximability. Empirical results on four binary datasets demonstrate that applying the WRC-Test improves robustness metrics (e.g., LOF, ) and validity for state-of-the-art CEGAs, supporting the practical utility of aligning explanation generation with the learning process.

Abstract

Counterfactual explanation generation is a powerful method for Explainable Artificial Intelligence. It can help users understand why machine learning models make specific decisions, and how to change those decisions. Evaluating the robustness of counterfactual explanation algorithms is therefore crucial. Previous literature has widely studied the robustness based on the perturbation of input instances. However, the robustness defined from the perspective of perturbed instances is sometimes biased, because this definition ignores the impact of learning algorithms on robustness. In this paper, we propose a more reasonable definition, Weak Robust Compatibility, based on the perspective of explanation strength. In practice, we propose WRC-Test to help us generate more robust counterfactuals. Meanwhile, we designed experiments to verify the effectiveness of WRC-Test. Theoretically, we introduce the concepts of PAC learning theory and define the concept of PAC WRC-Approximability. Based on reasonable assumptions, we establish oracle inequalities about weak robustness, which gives a sufficient condition for PAC WRC-Approximability.
Paper Structure (17 sections, 3 theorems, 26 equations, 3 figures, 1 table, 1 algorithm)

This paper contains 17 sections, 3 theorems, 26 equations, 3 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related works
  3. Preliminaries
  4. Learning Theory
  5. Counterfactual Explanations
  6. Weak Robust Compatibility and WRC-Test
  7. The Mathematical Formulation of CEGAs
  8. Defining the Robust Compatibility Between LAs and CEGAs
  9. Generating Robust Counterfactuals using WRC-Test
  10. A Sufficient Condition for PAC WRC- Approximability
  11. PAC WRC-Approximability
  12. Assumptions
  13. Theoretical Results
  14. Experiments
  15. Experimental Setup
  16. ...and 2 more sections

Key Result

Theorem 1

Let $\mathcal{H}=\mathcal{H}_\gamma,\mathcal{X}=[0,1]^{l+1}$. Assume that the learning problem satisfies the Tsybakov’s noise condition (eqTSY) and the CEGA $\mathcal{C}$ is induced by the metric $d(\cdot,\cdot)$ on $\mathcal{X}=[0,1]^{l+1}$. If hypothesis $h_T$ satisfies $R_{\mathcal{D}}(h_T)-R_{\m holds with probability of at least $1-\delta$.

Figures (3)

  • Figure 1: Illustration of the intuition behind the notion of SRC, where $f_T$ represents the classification boundary of classifier $h_T$.
  • Figure 2: The shape of the decision boundary results in a substantial distance between generated counterfactuals regardless of how small the perturbation in $x$ may be. However, the discrepancy in explanatory strength remains relatively insignificant. Therefore, if robustness is assessed using Definition \ref{['defSRC']}, the accuracy and robustness of CEGAs are contradictory. Nevertheless, adopting Definition \ref{['defWRC']} for assessing robustness can circumvent such contradictions.
  • Figure 3: Another Application of WRC: The numerically integrated WRC, estimated through computational methods, serves as a crucial reference metric for selecting a more robust CEGA.

Theorems & Definitions (13)

  • Definition 1: CEGA
  • Definition 2: CEGAs induced by $d(\cdot,\cdot)$
  • Remark 1
  • Definition 3: Strong Robust Compatibility
  • Remark 2
  • Remark 3
  • Definition 4: Weak Robust Compatibility
  • Definition 5: Discrete WRC
  • Definition 6: WRC-Test
  • Definition 7: PAC WRC-Approximable
  • ...and 3 more