Benchmarking Instance-Centric Counterfactual Algorithms for XAI: From White Box to Black Box

TL;DR

This study investigates the impact of machine learning models on the generation of counterfactual explanations by conducting a benchmark evaluation over three different types of models: a decision tree, a random forest, and a neural network.

Abstract

This study investigates the impact of machine learning models on the generation of counterfactual explanations by conducting a benchmark evaluation over three different types of models: a decision tree (fully transparent, interpretable, white-box model), a random forest (semi-interpretable, grey-box model), and a neural network (fully opaque, black-box model). We tested the counterfactual generation process using four algorithms (DiCE, WatcherCF, prototype, and GrowingSpheresCF) in the literature in 25 different datasets. Our findings indicate that: (1) Different machine learning models have little impact on the generation of counterfactual explanations; (2) Counterfactual algorithms based uniquely on proximity loss functions are not actionable and will not provide meaningful explanations; (3) One cannot have meaningful evaluation results without guaranteeing plausibility in the counterfactual generation. Algorithms that do not consider plausibility in their internal mechanisms will lead to biased and unreliable conclusions if evaluated with the current state-of-the-art metrics; (4) A counterfactual inspection analysis is strongly recommended to ensure a robust examination of counterfactual explanations and the potential identification of biases.

Figures (10)

• Figure 1: Counterfactual generation graph: Each point in the graph contains a different condition of an applicant, including age, credit amount, credit history, etc. A generated counterfactual contains the different conditions from the input of an applicant (point 0) that lead to a loan-approved result (orange dots). For instance, point $c$ indicates the applicant could be granted a loan if the income could be increased by 6%. Points $A$ and $B$ are the conditions for the applicant to remain loan rejected.
• Figure 2: Impact of different norms in sparsity. L$_1$-norm promotes sparseness because of its diamond shape function. The intersection of a vector with one of the function's corners will lead to a sparse result, whereas in the figure, only the $x$ coordinate will have a value different from $0$. This is not true for the L$_2$-norm due to its circular shape.
• Figure 3: The different counterfactual candidates for a data instance $x$. According to Watcher Watcher2018, counterfactual $\alpha$ is the best candidate because it has the shortest Euclidean distance to $x$. Other researchers believe that counterfactual instance $\Psi$ is the best option because it gives a feasible path from $x$ to $\Psi$Poyiadzi2020. Counterfactual $\beta$ is another candidate of poor quality because it lies in a less defined region of the decision boundary.
• Figure 4: Experimental design. Panel a. shows the five datasets used. Panel b. shows the three types of predictive models trained for each dataset. Panels c. and d. show the explainable counterfactual algorithms used and the corresponding results (the counterfactual explanations). Panel e. shows the metrics used to evaluate the generated counterfactuals (quantitative analysis). Panel f. shows the counterfactual inspection analysis used to assess the quality of the generated counterfactual explanations (counterfactual inspection).
• Figure 5: L$_1$ Norm of the generated counterfactual explanations across the different machine learning algorithms: DT corresponds to a Decision Tree, RF to a Random Forest, and NN to a Neural Network.