Prediction-powered Generalization of Causal Inferences

TL;DR

This work tackles the external validity problem of generalizing causal effects from randomized trials to target populations with different covariate distributions. It introduces prediction-powered estimators that fuse trial data with predictive models trained on observational data, without imposing strong OS assumptions, and derives theoretical MSE insights to explain when these methods help. Two main approaches are proposed: additive bias correction (ABC), which learns a bias function from the trial to polish OS predictions, and augmented outcome modeling (AOM), which incorporates OS-based predictors as covariates or representations; both can be implemented with regression-based estimators or doubly-robust schemes. Synthetic experiments across thousands of DGPs show that the OS-augmented methods improve generalization when OS quality is high and remain robust when OS is biased or confounded, offering a practical path to more reliable generalization in medicine and related fields.

Abstract

Causal inferences from a randomized controlled trial (RCT) may not pertain to a target population where some effect modifiers have a different distribution. Prior work studies generalizing the results of a trial to a target population with no outcome but covariate data available. We show how the limited size of trials makes generalization a statistically infeasible task, as it requires estimating complex nuisance functions. We develop generalization algorithms that supplement the trial data with a prediction model learned from an additional observational study (OS), without making any assumptions on the OS. We theoretically and empirically show that our methods facilitate better generalization when the OS is high-quality, and remain robust when it is not, and e.g., have unmeasured confounding.

Figures (14)

• Figure 1: Age influences both selection into the trial and the outcome, inducing confounding bias between the population-level mean potential outcomes in the trial and target populations.
• Figure 2: A biased predictor $f_a (X)$ can still capture higher order polynomials, making its bias $b_a (X)$ "easier" to learn than $g_a (X)$.
• Figure 3: Data-generating process used in simulated experiments. ( Left.) $X$ (observed) induces confounding by trial participation. ( Right.) In the observational study, there is hidden confounding for treatment assignment due to $U$ (unobserved).
• Figure 4: 100 different set of data-generating functions are sampled for each $(l_x^{\textnormal{FOM}_1}, \alpha_u^{\textnormal{PA}}, n_1)$. We plot the RMSE averaged over 100 scenarios. Results are reported for four different numbers of polynomial features used to fit the underlying regression functions (if any).
• Figure 5: Convention same as \ref{['fig:synthetic-rmse']}. The observational predictor is not trained on any data but generates i.i.d. noise for all $X$.

Theorems & Definitions (16)

• Proposition 3.0
• Theorem 3.1
• Lemma 4.0
• Theorem 4.1
• Lemma 4.1: Adopted from wainwright2019high
• Lemma 4.1
• Proposition 1.0
• proof
• Theorem 1.1
• proof