Evidence Aggregation for Treatment Choice

Abstract

Consider a planner who has limited knowledge of the policy's causal impact on a certain local population of interest due to a lack of data, but does have access to the publicized intervention studies performed for similar policies on different populations. How should the planner make use of and aggregate this existing evidence to make her policy decision? Following Manski (2020; Towards Credible Patient-Centered Meta-Analysis, \textit{Epidemiology}), we formulate the planner's problem as a statistical decision problem with a social welfare objective, and solve for an optimal aggregation rule under the minimax-regret criterion. We investigate the analytical properties, computational feasibility, and welfare regret performance of this rule. We apply the minimax regret decision rule to two settings: whether to enact an active labor market policy based on 14 randomized control trial studies; and whether to approve a drug (Remdesivir) for COVID-19 treatment using a meta-database of clinical trials.

Figures (4)

• Figure 1: Functional form of $\eta(a)$. The function $\eta(a)$ is strictly increasing and convex and $\eta(0)$ is approximately equal to 0.17.
• Figure 2: The black, red, and blue dots denote $\bm{w}_{\text{minimax}}$, $\bm{w}_{\text{MSE}}$, and $\bm{w}_{\text{HB}}$ for $C=1.0$.
• Figure 3: The black, red, and blue circles denote $\bm{w}_{\text{minimax}}$, $\bm{w}_{\text{MSE}}$, and $\bm{w}_{\text{HB}}$, respectively. The horizontal axis measures the Euclidean distance between $x_k$ and $x_0$, $\|x_k - x_0\|$. The size of the plotted circle is proportional to the precision of the estimates, i.e., a smaller $\hat{\sigma}_k$ corresponds to a larger circle.
• Figure 7: Heatmap of $\hat{\tau}_0(\bm{w}_{\text{minimax}})$ based on the values of $\hat{\tau}_0(\bm{w}_{\text{minimax}})$ computed at each $x_0$. Dark red areas correspond to regions of $x_0$ such that $\hat{\tau}_0(\bm{w}_{\text{minimax}})$ is positive and large. The white area corresponds to the region of $x_0$ such that $\hat{\tau}_0(\bm{w}_{\text{minimax}})$ is negative or near zero. The grey line is the boundary that separates the regions of positive and negative $\hat{\tau}_0(\bm{w}_{\text{minimax}})$. We also plot the covariate values of the meta-sample with the sizes of the plotted circles being proportional to the precision of the estimates.

Theorems & Definitions (20)

• Example 1: The space of $\bm{\tau}$ spanned by the meta-regressions
• Example 2: The class of constant variations
• Remark 1
• Definition 1: Linear aggregation rules
• Theorem 1
• Remark 2
• Theorem 2
• Remark 3
• Remark 4
• Remark 5