Moving toward best practice when using propensity score weighting in survey observational studies
Yukang Zeng, Fan Li, Guangyu Tong
TL;DR
This work addresses how to perform causal inference with propensity score weighting in complex survey data by presenting a unified balancing-weights framework that integrates survey weights into both the propensity score model and the outcome regression, yielding a general PWATE τ_h with tilting function h(X). It derives a closed-form, sandwich-variance-based inference procedure and extends weighting with three augmented estimators (MOM, CVR, WET) to exploit outcome models while preserving population-targeted inference. Through extensive simulations across overlap regimes and misspecifications, the authors demonstrate that using survey-weighted propensity scores (W.PS) consistently improves bias and coverage, and that the weighted regression augmented estimator (WET) with W.PS offers the most robust and efficient performance, especially for the overlap-population estimand PATO. The methods are illustrated with two case studies (ECLS-K and MEPS), showing practical gains in covariate balance and precision, and the authors provide practical guidelines and an R package (PSweight) to facilitate adoption in survey-based observational studies. Overall, the paper delivers a principled, generalizable approach for population-level causal inference under complex survey designs, with clear recommendations for weighting, augmentation, and variance estimation. Key contributions include a unified PWATE identification under survey designs, a suite of augmented estimators with closed-form variance, and empirical evidence favoring W.PS and WET across diverse scenarios, making these methods directly applicable to policy-relevant observational analyses in health services research and beyond.
Abstract
Propensity score weighting is a common method for estimating treatment effects with survey data. The method is applied to minimize confounding using measured covariates that are often different between individuals in treatment and control. However, existing literature does not reach a consensus on the optimal use of survey weights for population-level inference in the propensity score weighting analysis. Under the balancing weights framework, we provided a unified solution for incorporating survey weights in both the propensity score of estimation and the outcome regression model. We derived estimators for different target populations, including the combined, treated, controlled, and overlap populations. We provide a unified expression of the sandwich variance estimator and demonstrate that the survey-weighted estimator is asymptotically normal, as established through the theory of M-estimators. Through an extensive series of simulation studies, we examined the performance of our derived estimators and compared the results to those of alternative methods. We further carried out two case studies to illustrate the application of the different methods of propensity score analysis with complex survey data. We concluded with a discussion of our findings and provided practical guidelines for propensity score weighting analysis of observational data from complex surveys.