Empirically Understanding the Value of Prediction in Allocation

TL;DR

The paper tackles the question of when investing in prediction pays off in allocating scarce social resources. It introduces an empirical framework and the rvP toolkit that allow practitioners to compare prediction against capacity expansion and treatment improvements using domain-specific data, rather than relying on stylized theory. Through two case studies—German employment services and Ethiopian cash-transfer targeting—the authors show how the relative value of prediction depends on factors like capacity constraints, harm-versus-benefit considerations, and baseline data coverage, and they demonstrate how targeted data collection can be more efficient when improvements are focused near decision thresholds. The work provides actionable insights and replication resources, enabling policymakers to quantify welfare gains from prediction and to tailor investments to their specific context and budget. Overall, the contribution is a practical, data-driven approach to meta-design choices in allocation problems, expanding the toolkit for evaluating where predictive investments yield the strongest welfare returns.

Abstract

Institutions increasingly use prediction to allocate scarce resources. From a design perspective, better predictions compete with other investments, such as expanding capacity or improving treatment quality. Here, the big question is not how to solve a specific allocation problem, but rather which problem to solve. In this work, we develop an empirical toolkit to help planners form principled answers to this question and quantify the bottom-line welfare impact of investments in prediction versus other policy levers such as expanding capacity and improving treatment quality. Applying our framework in two real-world case studies on German employment services and poverty targeting in Ethiopia, we illustrate how decision-makers can reliably derive context-specific conclusions about the relative value of prediction in their allocation problem. We make our software toolkit, rvp, and parts of our data available in order to enable future empirical work in this area.

Figures (11)

• Figure 1: (a) Illustrates the task of identifying jobseekers at risk of long-term unemployment (purple) under a fixed capacity constraint. The employment office observes only imperfect predictions of risk, so the ranking of individuals is uncertain. (b) Increasing capacity expands the set of individuals who can be served. (c) Improving prediction sharpens the ranking, allowing limited resources to be targeted more effectively toward those truly at risk.
• Figure 2: Comparison of prediction improvements versus capacity expansion for targeting long-term unemployment risk ($\alpha = \beta = 0.15$). We simulate improving predictions for jobseekers over 35 years with missing employment history versus expanding screening capacity (assuming $4$ hours per slot). (a) Welfare difference as a function of RMSE reduction in the subgroup assuming that collecting missing employment information costs $1$ hour per jobseeker. The yellow line marks break-even. (b) Welfare difference when prediction improvements are concentrated on subgroup members near the decision threshold ($10\%$ bandwidth).
• Figure 3: Equivalent capacity expansion (in caseworker hours) that would generate the same welfare gain as each level of RMSE improvement.
• Figure 4: Gain from capacity expansion and uniform prediction improvement under different harm-to-benefit ratios $\tfrac{h}{b}$. All panels use $\beta = 0.25$, $\alpha = 0.01$, and allocate until capacity is exhausted while excluding individuals with negative expected utility.
• Figure 5: Ratio of welfare gains ($\beta = 0.25$, $\alpha = 0.01$) from prediction improvement (x-axis: RMSE reduction at $\tfrac{h}{b}=2$) versus harm reduction (y-axis: reduction in $\tfrac{h}{b}$). Heatmap values are truncated at $0.2$ and $5.0$.

Theorems & Definitions (2)

• Definition 3.1: Allocation Problem
• Definition 4.1: Policy Lever