The Impact of Machine Learning Uncertainty on the Robustness of Counterfactual Explanations

TL;DR

It is found that even small reductions in model accuracy can lead to large variations in the generated counterfactuals on average and on individual instances, underscore the need for uncertainty-aware explanation methods in domains such as finance and the social sciences.

Abstract

Counterfactual explanations are widely used to interpret machine learning predictions by identifying minimal changes to input features that would alter a model's decision. However, most existing counterfactual methods have not been tested when model and data uncertainty change, resulting in explanations that may be unstable or invalid under real-world variability. In this work, we investigate the robustness of common combinations of machine learning models and counterfactual generation algorithms in the presence of both aleatoric and epistemic uncertainty. Through experiments on synthetic and real-world tabular datasets, we show that counterfactual explanations are highly sensitive to model uncertainty. In particular, we find that even small reductions in model accuracy - caused by increased noise or limited data - can lead to large variations in the generated counterfactuals on average and on individual instances. These findings underscore the need for uncertainty-aware explanation methods in domains such as finance and the social sciences.

Figures (12)

• Figure 1: (a): The impact of noise level in model accuracy for mock dataset 1. The uncertainty added in this example is purely aleatoric (b): The impact of noise on the model's decision boundary for dataset 1. As the noise increases the decision boundary becomes more complicated and consequently it increases the error on the CE.
• Figure 2: The distribution of the normalised $\ell_1$–difference of the CE from the ground noise level, for mock dataset 2 (10 continuous predictive features). Each box spans Q1–Q3, with a line at the median; whiskers extend to 1.5 times IQR, and outliers beyond are shown as circles.
• Figure 3: The distribution of the normalised $\ell_1$–difference of the CE from the ground noise level, for mock dataset 1 (2 continuous predictive features). Each box spans Q1–Q3, with a line at the median; whiskers extend to 1.5 times IQR, and outliers beyond are shown as circles.
• Figure 4: $\ell_1$ distance difference from the ground state with minimal noise for various combinations of classification algorithms and CE methods again model accuracy. Only combinations with CE completeness $>10\%$ are included. Each panel shows the CE of test set of the classification exercise split into FN, all and TN.
• Figure 5: The distribution of the normalized $\ell_1$ norm for different ML and CE methods in the presence of both aleatoric and epistemic uncertainty. The construction of the boxes, whiskers and outliers is the same as in Fig. \ref{['fig:l1_1_page']}. The dashed red line depicts the baseline median CE distance for the case with aleatoric uncertainty present only.