On the Definition and Detection of Cherry-Picking in Counterfactual Explanations

TL;DR

This work formalizes cherry-picking in counterfactual explanations by defining an admissible explanation space $_{\uF, }(x)$ and a utility-based ranking $ ext{rank}_x$ under which an explanation is cherry-picked if it is not top-ranked. It examines detectability under three access levels—full procedural, partial procedural, and explanation-only—finding that detection is severely limited in practice due to the multiplicity of valid explanations and the inherent variability of explanation generation. Empirical demonstrations using DiCE show that randomness and method choices can induce greater variation in proximity, plausibility, and sparsity than the cherry-picking signal, undermining post-hoc detection. The authors argue for ex ante safeguards—reproducibility, standardisation, and procedural constraints—over post-hoc detection and provide concrete recommendations for algorithm developers, explanation providers, and auditors to mitigate the risk of selective disclosure. Overall, the paper highlights a structural challenge in XAI governance and calls for domain-specific specifications to reduce opportunities for cherry-picking in high-stakes decisions.

Abstract

Counterfactual explanations are widely used to communicate how inputs must change for a model to alter its prediction. For a single instance, many valid counterfactuals can exist, which leaves open the possibility for an explanation provider to cherry-pick explanations that better suit a narrative of their choice, highlighting favourable behaviour and withholding examples that reveal problematic behaviour. We formally define cherry-picking for counterfactual explanations in terms of an admissible explanation space, specified by the generation procedure, and a utility function. We then study to what extent an external auditor can detect such manipulation. Considering three levels of access to the explanation process: full procedural access, partial procedural access, and explanation-only access, we show that detection is extremely limited in practice. Even with full procedural access, cherry-picked explanations can remain difficult to distinguish from non cherry-picked explanations, because the multiplicity of valid counterfactuals and flexibility in the explanation specification provide sufficient degrees of freedom to mask deliberate selection. Empirically, we demonstrate that this variability often exceeds the effect of cherry-picking on standard counterfactual quality metrics such as proximity, plausibility, and sparsity, making cherry-picked explanations statistically indistinguishable from baseline explanations. We argue that safeguards should therefore prioritise reproducibility, standardisation, and procedural constraints over post-hoc detection, and we provide recommendations for algorithm developers, explanation providers, and auditors.

Figures (6)

• Figure 1: Toy example of multiple counterfactual instances ($e_1, e_2, e_3$) for the same data instance $x$. The corresponding counterfactual explanations are the red arrows $\mathrm{cfe}_i = e_i - x$, each crossing the blue decision boundary. The table reports the feature values for $x$ and each $e_i$, and the feature changes encoded by each $\mathrm{cfe}_i$.
• Figure 2: Proximity of optimal vs cherry-picked explanation space for DiCE random with 10 different seeds.
• Figure 3: Mean sparsity of explanations generated by DiCE (random), comparing a baseline run with 50 different seeds for both the model and the counterfactual generation. Results are shown for All, where all features may be edited, and Restricted, where edits are limited to a predefined restricted feature set (non-sensitive features only).
• Figure 4: Mean plausibility versus proximity on the Adult dataset (50 seeds per group). Colours indicate the source of variation: Baseline uses a fixed seed (42), while Model and CF represent variations where the model seed or counterfactual initialisation seed are randomised, respectively. The second term denotes the feature constraints: All permits edits to any feature, whereas Restricted limits edits to non-sensitive features only.
• Figure 5: Mean Proximity and Plausibility of cherry-picked vs. alternative explanation settings for the Adult dataset.

Theorems & Definitions (12)

• Example 1.1
• Definition 3.1
• Example 3.1
• Example 3.2
• Example 4.1
• Example 4.2
• Proposition 4.1
• proof
• Remark 4.1
• Proposition 4.2