Feature Responsiveness Scores: Model-Agnostic Explanations for Recourse

TL;DR

This work tackles whether explanations given to decision subjects truly enable recourse in high-stakes settings. It introduces feature responsiveness scores, a model-agnostic measure of the probability that an intervention on a feature leads to a target prediction under defined actionability constraints, captured via reachable sets. The authors develop sampling and enumeration methods to compute these scores, demonstrate that standard SHAP/LIME explanations often highlight unresponsive or immutable features, and show that responsiveness-based explanations improve recourse while flagging fixed predictions. The results, validated on multiple lending datasets, highlight the practical need to incorporate actionability, degree, and direction information into explanations, and provide an open-source library to compute responsiveness scores for real-world deployment.

Abstract

Consumer protection rules require companies that deploy models to automate decisions in high-stakes settings to explain predictions to decision subjects. These rules are motivated, in part, by the belief that explanations can promote recourse by revealing information that decision subjects can use to contest or overturn their predictions. In practice, companies provide individuals with a list of principal reasons based on feature importance derived from methods like SHAP and LIME. In this work, we show how common practices can fail to provide recourse and propose to highlight features based on their responsiveness -- the probability that a decision subject can attain a target prediction through an arbitrary intervention on the feature. We develop efficient methods to compute responsiveness scores for any model and actionability constraints. We show that standard practices in lending can undermine decision subjects by highlighting unresponsive features and explaining predictions that are fixed.

Figures (14)

• Figure 1: Feature-highlighting explanations for a person denied credit by an XGBoost model on the givemecredit dataset in \ref{['Sec::Experiments']}. We show explanations that highlight up to 4 features with the largest SHAP scores (left) and responsiveness scores (right). As shown, an explanation built with SHAP highlights features that the person cannot change (e.g., Age, HistoryOfLatePayment, NumberOfDependents) or unresponsive (CreditUsage, which can be changed but would not lead to a target prediction). In contrast, an explanation built with responsiveness scores highlights the only 2 features lead to a desired prediction: MonthlyIncome and MultipleCreditLines.
• Figure 2: Standard methods for recourse provision return the closest action that leads to a target prediction ${\bm{a}^\textrm{opt}}$. Our method estimates the proportion of actions on each feature that lead to a target prediction. Here, $\mu_1 = \tfrac{3}{4}$ and $\mu_2=\tfrac{1}{4}$ because $\bm{x}$ can attain a target prediction through 3/4 actions on $x_1$, or 1/4 actions on $x_2$.
• Figure 3: Stylized example showing how to compute responsiveness scores for a classification model with three features n_loans, guarantor and age. The reachable set $R_j(\bm{x})$ all points that can be attained from $\bm{x} = (3,0,24)$ by intervening on feature $j$, and $R_3(\bm{x}) = \varnothing$ because age is immutable. Given a model $f$, we compute the responsiveness score of each feature by querying its predictions over points in their reachable set $R_j(\bm{x})$.
• Figure 4: Responsiveness of features for individuals who are denied credit by the $\mathsf{LR}$ model on the fico dataset according to absolute feature attribution rank using the original feature attribution method, its action-aware variant and RESP. For each method, we report the proportion of individuals with at least one responsive intervention on a feature with the $k$-th largest score ($k$-th ranked feature). Features must have non-zero score to be included in a "rank."
• Figure 5: Responsiveness of features for individuals who are denied credit by the $\mathsf{LR}$ model on the fico dataset according to absolute feature attribution rank using the original feature attribution method, its action-aware variant and RESP. For each method, we report the proportion of individuals with at least one responsive intervention on a feature with the $k$-th largest score ($k$-th ranked feature). Features must have non-zero score to be included in a "rank."

Theorems & Definitions (7)

• Definition 1
• Definition 2
• Definition 3
• Remark 1
• Remark 2
• Remark 3
• proof