Counterfactual Analysis and Target Setting in Benchmarking

TL;DR

This work addresses the interpretability gap in Data Envelopment Analysis (DEA) by introducing counterfactual explanations and targets that identify near-original input/output adjustments to achieve higher efficiency. The authors formulate a bilevel optimization where a cost-minimizing upper level selects a counterfactual input profile while a lower-level DEA model evaluates the resulting efficiency, and they derive a tractable single-level reformulation using $F=1/E$ and $\beta^k=\lambda^k/E$, solvable as MILP or mixed-integer convex quadratic programs. The key contributions include generalizing DEA target literature, integrating cost-of-change concepts from interpretable ML, and providing a practical banking-branch application that yields actionable, near-minimal input changes. The approach supports learning, decision making, and incentives design in operational and regulatory contexts, and offers extensions to other efficiency measures, returns-to-scale settings, and group-level counterfactuals for systemic insights.

Abstract

Data Envelopment Analysis (DEA) allows us to capture the complex relationship between multiple inputs and outputs in firms and organizations. Unfortunately, managers may find it hard to understand a DEA model and this may lead to mistrust in the analyses and to difficulties in deriving actionable information from the model. In this paper, we propose to use the ideas of target setting in DEA and of counterfactual analysis in Machine Learning to overcome these problems. We define DEA counterfactuals or targets as alternative combinations of inputs and outputs that are close to the original inputs and outputs of the firm and lead to desired improvements in its performance. We formulate the problem of finding counterfactuals as a bilevel optimization model. For a rich class of cost functions, reflecting the effort an inefficient firm will need to spend to change to its counterfactual, finding counterfactual explanations boils down to solving Mixed Integer Convex Quadratic Problems with linear constraints. We illustrate our approach using both a small numerical example and a real-world dataset on banking branches.

Figures (7)

• Figure 1: Directional search for an alternative production plan to $(\bm{x},\bm{y})$ along $(\bm{d}_x,\bm{d}_y)$ using (DIR)
• Figure 2: Directional search in Interactive Benchmarking software. Real-world dataset of bank branches in Section \ref{['sec:num']}
• Figure 3: Counterfactual explanations for firms 3 and 4 in Tables \ref{['tab:sol']} and \ref{['tab:sol2']} respectively imposing $E^{*}=0.8$ and different cost functions
• Figure 4: Counterfactual Explanations for firm 238 with Problem \ref{['eq:cedeafinal']} and desired efficiency $E^{*}=1$.
• Figure 5: The inputs that change when we impose a desired efficiency of $E^{*}=1$