Omitted-Variable Sensitivity Analysis for Generalizing Randomized Trials

Abstract

Randomized controlled trials (RCTs) yield internally valid causal effect estimates, but generalizing these results to target populations with different characteristics requires an untestable selection ignorability assumption: conditional on observed covariates, trial participation must be independent of potential outcomes. This assumption fails when unobserved effect modifiers are distributed differently between trial and target populations. We develop a sensitivity analysis framework for trial generalization grounded in omitted variable bias (OVB). Our key theoretical contribution is an exact decomposition showing that external-validity bias equals moderation strength $\times$ moderator imbalance: (i) how strongly an unobserved variable shifts the treatment effect, times (ii) how differently that variable is distributed across populations after covariate adjustment. We introduce scale-free sensitivity parameters based on partial $R^2$ values, enabling closed-form bounds and benchmarking against observed covariates -- practitioners can assess whether conclusions would change if an unobserved moderator were "as strong as" a particular observed variable. Simulations demonstrate that our bounds achieve nominal coverage and remain conservative under model misspecification, while comparisons with alternative sensitivity frameworks highlight the interpretive advantages of the OVB decomposition.

Figures (11)

• Figure 1: Causal structure for trial generalization with an unobserved effect modifier $U$. Observed variables ($X$, $A$, $Y$, $S$) are shaded; latent $U$ is unshaded with a dashed border. The dashed edge $U \dashrightarrow S$ indicates that $U$'s distribution differs between trial ($S=1$) and target ($S=0$) populations, i.e., $P(U \mid X, S=1) \neq P(U \mid X, S=0)$. If $U$ also modifies treatment effects ($U \to Y$ interaction with $A$), selection ignorability fails and standard transport estimators are biased.
• Figure 2: Monte Carlo coverage vs. moderation bound $\Gamma$ in a linear-Gaussian DGP. Coverage is 0% for $\Gamma < \Gamma^*$ and jumps to 100% at $\Gamma=\Gamma^*$ (green dashed).
• Figure 3: Coverage comparison: bias envelope only (solid) vs. full confidence interval combining sensitivity bounds with bootstrap uncertainty (dashed). The full CI achieves valid coverage even at $\Gamma=0$ due to sampling variability.
• Figure 4: Benchmarking scatter plot. Each observed covariate is plotted by its partial $R^2$ for selection ($x$-axis) and treatment effect ($y$-axis); the dashed line is the sign-reversal robustness threshold.
• Figure 5: OVB sensitivity envelope for a single simulation. The baseline estimate $\hat{\tau}_X$ is biased for the true TATE $\tau^*$, and the sensitivity interval expands with $\Gamma$.

Theorems & Definitions (12)

• Proposition 3.3: Identification under selection ignorability
• Remark 3.4: Alternative estimators
• Example 4.2: Unmeasured effect modifiers
• Remark 4.4: Interpretation
• Lemma 4.5: External-validity OVB identity
• Corollary 4.6: Raw sensitivity interval
• Remark 5.3: When is constant imbalance reasonable?
• Theorem 5.4: Partial-$R^2$ bound for external-validity bias
• Proposition 6.1: Robustness value
• Example 6.2: Benchmark interpretation