A Cautionary Tale on Integrating Studies with Disparate Outcome Measures for Causal Inference

TL;DR

This work tackles causal inference when integrating datasets with non-overlapping outcomes by introducing an outcome-link model $\nu^t_Y(x)=\alpha(x)\nu^t_W(x)+\beta(x)$. It develops three linkage scenarios with varying prior knowledge about $\alpha$ and $\beta$, and derives semiparametric efficiency bounds and finite-sample risk under each. The authors show that asymptotic efficiency gains are only achieved under the strongest, fully known-link, while milder linkages can still yield finite-sample gains but may entail bias if mis-specified. Through simulations and a MOUD case study using XBOT (SOWS) and POATS (COWS), they illustrate the practical tradeoffs and provide guidance on when auxiliary data with disparate outcomes can improve estimation. The framework highlights the balance between leveraging auxiliary information and the risk of misspecification, with concrete guidance for researchers navigating non-overlapping outcomes in modern data integration.

Abstract

Data integration approaches are increasingly used to enhance the efficiency and generalizability of studies. However, a key limitation of these methods is the assumption that outcome measures are identical across datasets -- an assumption that often does not hold in practice. Consider the following opioid use disorder (OUD) studies: the XBOT trial and the POAT study, both evaluating the effect of medications for OUD on withdrawal symptom severity (not the primary outcome of either trial). While XBOT measures withdrawal severity using the subjective opiate withdrawal scale, POAT uses the clinical opiate withdrawal scale. We analyze this realistic yet challenging setting where outcome measures differ across studies and where neither study records both types of outcomes. Our paper studies whether and when integrating studies with disparate outcome measures leads to efficiency gains. We introduce three sets of assumptions -- with varying degrees of strength -- linking both outcome measures. Our theoretical and empirical results highlight a cautionary tale: integration can improve asymptotic efficiency only under the strongest assumption linking the outcomes. However, misspecification of this assumption leads to bias. In contrast, a milder assumption may yield finite-sample efficiency gains, yet these benefits diminish as sample size increases. We illustrate these trade-offs via a case study integrating the XBOT and POAT datasets to estimate the comparative effect of two medications for opioid use disorder on withdrawal symptoms. By systematically varying the assumptions linking the SOW and COW scales, we show potential efficiency gains and the risks of bias. Our findings emphasize the need for careful assumption selection when fusing datasets with differing outcome measures, offering guidance for researchers navigating this common challenge in modern data integration.

Figures (2)

• Figure 1: MOUD Results. (a) Scatter plot showing the relationship between SOWS and COWS. (b) Treatment effect estimates of MOUD on withdrawal symptoms. Point estimates and corresponding 95% confidence intervals for $\hat{\theta}_0$, $\hat{\theta}_a$ and $\hat{\theta}_b$. (c) Assessing the sensitivity of $\hat{\theta}_a$ to the different values of $\alpha$ in the range 50% above and below the original guess of $\alpha = 0.61$.
• Figure 2: Simulation Study Results. Mean squared error rates for three different estimators $\hat{\theta}_{0}$, $\hat{\theta}_{a}$ and $\hat{\theta}_{b}$ based on $R^*_0$, $R^*_a$, and $R^*_b$

Theorems & Definitions (12)

• Remark 1: On Assumption \ref{['eq: y_w']}
• Theorem 1: Efficiency bound using only primary data
• Theorem 2: Efficiency bound under known $\alpha(X)$ and $\beta(X)$
• Corollary 1
• Theorem 3: Efficiency bound under known $\beta(X)$ only
• Corollary 2
• Theorem 4: Efficiency bound under unknown $\alpha(X)$ and $\beta(X)$
• Corollary 3
• Theorem 5: Misspecification Bias
• Theorem 6: Error bound for $\hat{\mu}_{Y}$