Demystify Doubly-Robust Estimation: The Role of Overlap

TL;DR

This study examines the finite-sample performance of the doubly-robust estimator under varying covariate overlap and model misspecification. By deriving a finite-sample error decomposition $\Delta_{dr}=\frac{1}{N}\sum_i R_i^e R_i^y$ and conducting extensive simulations plus an applied example, the authors show that DR’s apparent protection in theory can deteriorate in practice, especially when overlap is poor and the outcome model is misspecified; the dominant influence of the outcome model grows as overlap worsens. They demonstrate that DR amplifies adverse effects from extreme inverse-propensity weights and that trimming or shifting inference to overlapping subpopulations (via trimming or overlap weighting) yields more robust, efficient estimates. The practical guidance emphasizes assessing overlap first and, if overlap is poor, adopting overlap-focused targets or calibrated weights rather than relying on DR to salvage inference. The fibroid registry application illustrates how overlap-aware methods can yield more stable and interpretable estimates, highlighting potential heterogeneity across subpopulations as a key consideration for causal analyses in observational data.

Abstract

The doubly-robust (DR) estimator is popular for evaluating causal effects in observational studies and is often perceived as more desirable than inverse probability weighting (IPW) or outcome modeling alone because it provides extra protection against model misspecification. However, double robustness is an asymptotic property that may not hold in finite samples. We investigate how the finite sample performance of the DR estimator depends on the degree of covariate overlap between comparison groups. Using analytical illustrations and extensive simulations under various scenarios with different degrees of covariate overlap and model specifications, we examine the bias and variance of the DR estimator relative to IPW and outcome modeling estimators. We find that: (i) specification of the outcome model has a stronger influence on the DR estimates than specification of the propensity score model, and this dominance increases as overlap decreases; (ii) with poor overlap, the DR estimator generally amplifies the adverse consequences of extreme weights (large bias and/or variance) regardless of model specifications, and is often inferior to both the IPW and outcome modeling estimators. As a practical guide, we recommend always first checking the degree of overlap in applications. In the case of poor overlap, analysts should consider shifting the target population to a subpopulation with adequate overlap via methods such as trimming or overlap weighting.

Figures (15)

• Figure 1: Scaled relative MAV of $R_i^e$ and $R_i^y$. Misspecification is incurred by wrong functional form (mistaking $X_3$ by $X_3^2$). The green line ($\Phi(l)$) indicates no change in relative scale. Large values at tails of panel (b) and the rest of panels (c) and (d) are delegated to the Supplementary Material.
• Figure 2: Scaled relative MAV of $R_i^e R_i^y$. Misspecification is incurred by wrong functional form (mistaking $X_3$ by $X_3^2$). The green line ($\Phi(l)$) indicates no change in relative scale. Large values at tails of panel (b) and the rest of panels (c) and (d) are delegated to the Supplementary Material.
• Figure 3: Distribution of estimated propensity scores by treatment for uterine fibroids.
• Figure A1: Scaled relative MAV of $R_i^e$ and $R_i^y$. Misspecification is incurred by omitting variables (omitting $X_3$ and $X_4$). The green line ($\Phi(l)$) indicates no change in relative scale. Large values at tails of panel (b) and the rest of panels (c) and (d) are delegated to Table \ref{['tab:A6']} and Figure \ref{['fig:A3']}.
• Figure A2: Scaled relative MAV of $R_i^e R_i^y$. Misspecification is incurred by omitting variables (omitting $X_3$ and $X_4$). The green line ($\Phi(l)$) indicates no change in relative scale. Large values at tails of panel (b) and the rest of panels (c) and (d) are delegated to Table \ref{['tab:A7']} and Figure \ref{['fig:A4']}.