Interpretable Model-Aware Counterfactual Explanations for Random Forest

TL;DR

This paper addresses the need for interpretable, actionable explanations in regulated domains by introducing a model-aware counterfactual framework for random forests. It leverages RF-GAP proximity to define a model-consistent similarity metric and constructs counterfactual trajectories that minimize input-space distance while meaningfully changing the prediction, explained via partition-crossing tallies. Empirically, the approach yields sparse, faithful explanations and comparable or superior interpretability to SHAP on MNIST and German Credit, providing more intuitive recourse for domain users. The work lays groundwork for applying case-based counterfactuals to high-stakes settings and suggests extensions to other ensemble models and regulated domains such as healthcare.

Abstract

Despite their enormous predictive power, machine learning models are often unsuitable for applications in regulated industries such as finance, due to their limited capacity to provide explanations. While model-agnostic frameworks such as Shapley values have proved to be convenient and popular, they rarely align with the kinds of causal explanations that are typically sought after. Counterfactual case-based explanations, where an individual is informed of which circumstances would need to be different to cause a change in outcome, may be more intuitive and actionable. However, finding appropriate counterfactual cases is an open challenge, as is interpreting which features are most critical for the change in outcome. Here, we pose the question of counterfactual search and interpretation in terms of similarity learning, exploiting the representation learned by the random forest predictive model itself. Once a counterfactual is found, the feature importance of the explanation is computed as a function of which random forest partitions are crossed in order to reach it from the original instance. We demonstrate this method on both the MNIST hand-drawn digit dataset and the German credit dataset, finding that it generates explanations that are sparser and more useful than Shapley values.

Figures (2)

• Figure 1: Counterfactual trajectory explanations on the MNIST dataset. a) A counterfactual trajectory shown for a digit '3' vs the label '8', plotted in a t-SNE embedding of the dataset. b) The same trajectory plotted in an MDS embedding of RF-GAP distances. c) Visualization of different counterfactual explanations that may be obtained by tallying random forest partition crossings. d) Comparison with other common ML explanation frameworks.
• Figure 2: Explanations for the German credit dataset. a) The counterfactual explanation for the instance in the test set with the lowest credit worthiness vs the instance with the highest credit worthiness. i) The counterfactual trajectory plotted in an embedding of the dataset using the RF-GAP distances from the predictive model. ii) The random forest partition tally integrated along the counterfactual trajectory, sorted by absolute value for the top 20 features. iii) The Shapley values for the same features in (ii). The values of the instance and its counterfactual for those features. b) A counterfactual relationship for an instance just shy of 'good' credit worthiness ($P(good)=0.49$) vs its closest 'class flip' ($P(good)=0.58$). c) Usefulness of counterfactuals, evaluated as frequency of class flips against number of features changed to that of an instance's counterfactual, for every instance in the test set ($n=500$).