Counterfactual Explanations with Probabilistic Guarantees on their Robustness to Model Change

TL;DR

The authors address the problem of counterfactual explanations losing validity when the underlying model changes. They introduce a probabilistic, model-agnostic framework for robustness to model change and the BetaRCE post-hoc method, which can wrap any base CFE generator to produce CFEs that are $(\delta, \alpha)$-robust. By modeling robustness with a Beta-Bernoulli approach and verifying it via bootstrap over an admissible model space, they provide interpretable hyperparameters and practical guarantees. Empirical results across diverse datasets and model changes show BetaRCE yields robust CFEs that remain close to base explanations and outperform several baselines in key quality metrics, supporting its potential for real-world deployment.

Abstract

Counterfactual explanations (CFEs) guide users on how to adjust inputs to machine learning models to achieve desired outputs. While existing research primarily addresses static scenarios, real-world applications often involve data or model changes, potentially invalidating previously generated CFEs and rendering user-induced input changes ineffective. Current methods addressing this issue often support only specific models or change types, require extensive hyperparameter tuning, or fail to provide probabilistic guarantees on CFE robustness to model changes. This paper proposes a novel approach for generating CFEs that provides probabilistic guarantees for any model and change type, while offering interpretable and easy-to-select hyperparameters. We establish a theoretical framework for probabilistically defining robustness to model change and demonstrate how our BetaRCE method directly stems from it. BetaRCE is a post-hoc method applied alongside a chosen base CFE generation method to enhance the quality of the explanation beyond robustness. It facilitates a transition from the base explanation to a more robust one with user-adjusted probability bounds. Through experimental comparisons with baselines, we show that BetaRCE yields robust, most plausible, and closest to baseline counterfactual explanations.

Figures (12)

• Figure 1: Our method BetaRCE post-hoc generates counterfactuals at desired levels of robustness to model change, having some probabilistic properties. First, the base CFE is generated using any base method, then BetaRCE is applied to move that CFE to a region satisfying $(\delta,\alpha)$-robustness .
• Figure 2: The average Empirical Robustness of counterfactuals generated by BetaRCE at $\alpha = 90\%$, with GrowingSpheres generating base CFEs. Red and green dashed lines are average lower and upper bounds of estimated $\alpha$ credible intervals (CI). The shaded areas in the back represent standard deviations. Horizontal yellow and gray lines show the base robustness of CFEs obtained with Dice and GrowingSpheres without BetaRCE applied. The plots with all four datasets and two BetaRCE base CFE generation methods (Dice and GrowingSpheres ) are available in App. \ref{['app:sec:cred-interv-the']}.
• Figure 3: Left: The average empirical robustness computed at various confidence ($\alpha$) and robustness ($\delta$) levels on the HELOC dataset, with $k$ set to $32$. Right: The impact of the number of $k$ estimators on the width of $90\%$ credible interval, computed on the HELOC dataset.
• Figure 4: Some priors of Beta distribution
• Figure 5: Max achievable $\delta$ values as a function of $k$ and $\alpha$.

Theorems & Definitions (9)

• Definition 1: Robust counterfactual
• Definition 2: Space of admissible models
• Definition 3: $\delta$-robust counterfactual
• Definition 4: $(\delta, \alpha)$-robust counterfactual
• Theorem 1
• Remark 1
• Theorem 2
• Lemma 3
• Lemma 4