Learning Decision Trees and Forests with Algorithmic Recourse

TL;DR

This work tackles learning tree-based models with guaranteed executable recourse actions by introducing a recourse-aware objective for classification trees and extending to random forests. It proposes a top-down greedy splitting algorithm that integrates adversarial recourse considerations, followed by a set-cover based relabeling step to enforce a recourse budget with approximation guarantees. The method, named Recourse-Aware Classification Tree (RACT), demonstrates higher recourse coverage than baselines while maintaining comparable predictive accuracy and efficiency, and it shows favorable action quality including lower cost and robust causal recourse validity. The approach enhances trustworthiness in critical decision tasks by coupling predictive performance with practical, executable recourse, and offers a tunable balance between accuracy and recourse through a hyperparameter, with clear directions for future extensions and potential societal impacts.

Abstract

This paper proposes a new algorithm for learning accurate tree-based models while ensuring the existence of recourse actions. Algorithmic Recourse (AR) aims to provide a recourse action for altering the undesired prediction result given by a model. Typical AR methods provide a reasonable action by solving an optimization task of minimizing the required effort among executable actions. In practice, however, such actions do not always exist for models optimized only for predictive performance. To alleviate this issue, we formulate the task of learning an accurate classification tree under the constraint of ensuring the existence of reasonable actions for as many instances as possible. Then, we propose an efficient top-down greedy algorithm by leveraging the adversarial training techniques. We also show that our proposed algorithm can be applied to the random forest, which is known as a popular framework for learning tree ensembles. Experimental results demonstrated that our method successfully provided reasonable actions to more instances than the baselines without significantly degrading accuracy and computational efficiency.

Figures (10)

• Figure 1: Examples of an input instance $\bm{x}$ and classification trees $h_0, h_1$. To get the loan approved from $h_0$, the instance $\bm{x}$ needs to just reduce "#ExistingLoans." In contrast, for the case of $h_1$, $\bm{x}$ needs to change "Purpose" or "Education," which are not executable easily.
• Figure 2: Running example of our top-down greedy splitting algorithm on $\mathcal{X} = \mathbb{R}^2$. Here, we have six labeled instances $(\bm{x}, y)$ as a training sample $S$, and blue (resp. red) indicates the desired class $y = +1$ (reps. undesired class $y = -1$). We assume that while the feature $x_1$ is immutable (i.e., can not be changed by an action), the feature $x_2$ is mutable (i.e., can be changed by an action). We also assume thresholds $B_1 = \{ b_1 \}$ and $B_2 = \{ b_2, b_3 \}$ for the features $x_1$ and $x_2$, respectively, and set the trade-off parameter as $\lambda = 1$. For each instance $\bm{x}$ that is located in the red region corresponding to $\hat{y} = -1$, the green line stands for the range where $\bm{x}$ can reach by some action $\bm{a}$ within a pre-defined cost budget $\varepsilon$. For each split condition $(d, b)$, we first determine predictive labels $\hat{y}_\mathrm{L}$ and $\hat{y}_\mathrm{R}$ by solving the inner problem of \ref{['eq:split']}, and then find a best split condition that minimizes our objective function $\Phi_{\lambda}$. Note that the objective value of each split condition $(d, b)$ can be computed in amortized constant time.
• Figure 3: Experimental results of baseline comparison. Figures in the top (resp. bottom) row present predictive accuracy (resp. recourse ratio). Our RACT attained higher recourse ratio than the baselines while keeping comparable accuracy on almost all the datasets.
• Figure 4: Sensitivity analyses of the trade-off parameter $\lambda$ with respect to the average AUC and recourse ratio. While the value ranges of $\lambda$ are different across the datasets, we can see that the recourse ratio (resp. AUC) was improved as $\lambda$ increased (resp. decreased) overall.
• Figure 5: Average feature importance of random forests leaned by each method in the performance comparison. We measured the importance score of each feature by averaging the number of times the feature is used among trees, and normalized the scores so that the sum equals to $1$. For each feature, "I" (resp. "M") stands for an immutable (resp. mutable) feature.

Theorems & Definitions (6)

• Proposition 3.1
• Proposition 3.2
• Proposition 3.3
• proof : Proof of \ref{['prop:algo']}
• proof : Proof of \ref{['prop:wpc']}
• proof : Proof of \ref{['prop:pac']}