Prediction without Preclusion: Recourse Verification with Reachable Sets

TL;DR

This work tackles the risk of fixed predictions arising from actionability constraints in ML-driven decisions like lending and hiring. It introduces recourse verification, a model-agnostic auditing framework that certifies whether a subject can reach a target prediction by moving within a reachable set $R_A = \{ \bm{x} + \bm{a} \mid \bm{a} \in A \}$, and it leverages a mixed-integer programming formulation to enumerate reachable points, with decomposition and interior-approximation options to manage complexity. The authors demonstrate, across three consumer-finance datasets, that substantial fractions of instances exhibit preclusion (No recourse) and that popular recourse tools can produce actions that violate actionability (loopholes) or fail to provide actions (blindspots). The paper provides a software package for practitioners to specify complex actionability constraints and perform recourse verification, enabling more reliable auditing and safer deployment of decision systems. Overall, recourse verification offers a principled, model-agnostic method to detect and mitigate preclusion, guiding better model design and policy decisions when actionability constraints are present.

Abstract

Machine learning models are often used to decide who receives a loan, a job interview, or a public benefit. Models in such settings use features without considering their actionability. As a result, they can assign predictions that are fixed $-$ meaning that individuals who are denied loans and interviews are, in fact, precluded from access to credit and employment. In this work, we introduce a procedure called recourse verification to test if a model assigns fixed predictions to its decision subjects. We propose a model-agnostic approach for recourse verification with reachable sets $-$ i.e., the set of all points that a person can reach through their actions in feature space. We develop methods to construct reachable sets for discrete feature spaces, which can certify the responsiveness of any model by simply querying its predictions. We conduct a comprehensive empirical study on the infeasibility of recourse on datasets from consumer finance. Our results highlight how models can inadvertently preclude access by assigning fixed predictions and underscore the need to account for actionability in model development.

Figures (4)

• Figure 1: Stylized classification task where the most accurate classifier on a dataset with $n^{-{}}_{} = 60$ negative examples and $n^{+{}}_{} = 90$ positive examples assigns a prediction without recourse to individuals with $(x_1, x_2) = (1,1)$. We predict $y = \texttt{\footnotesize{repay\_loan}}$ using two binary features $(x_1, x_2) = ({\texttt{\footnotesize{reapplicant}},\texttt{\footnotesize{{age}}}\geq\texttt{\footnotesize{60}}})$, which can only increase from 0 to 1. We denote the actions on each feature as $(a_1, a_2)$ and show the constraints they must obey in the action set. Given any model, we certify the responsiveness of its outputs for $(x_1, x_2)$ by checking its prediction for each point in the reachable set$R_A(x_1, x_2)$. In this case, 42 individuals with $(x_1, x_2) = (1,1)$ are assigned a prediction without recourse as $f(x_1, x_2) = 0{}$ for all $(x_1, x_2) \in R_A(1, 1)$.
• Figure 2: Reachable sets for each point in the heloc dataset, ordered from smallest to largest along the $x$-axis. We show predictions without recourse in red and highlight incorrectly predicted points in darker colors. As shown, predictions without recourse are prevalent across all reachable set sizes and can vary significantly between classifiers.
• Figure 3: $V$ denotes valid combinations of features. For these features, we wanted to produce actions that would reduce NetFractionRevolvingBurden for consumers. $E$ shows which points can be reached. For example, $[1, 1, 0, 0]$ represents point $[0, 0, 0]$ can be reached, and point $[1, 0, 0]$ can be reached, but no other points can be reached.
• Figure : $\mathsf{GetReachableSet}$

Theorems & Definitions (6)

• Definition 1
• Definition 2
• Definition 3
• Theorem 4: label=Thm::PrevalenceBasedRecourseRule
• Theorem 5: continues=Thm::PrevalenceBasedRecourseRule
• proof