Semiparametric Efficient Data Integration Using the Dual-Frame Sampling Framework

TL;DR

This work tackles the problem of integrating probability and non-probability samples when the non-probability inclusion mechanism is unknown. It develops a dual-frame, semiparametric theory and introduces two estimators: an efficient estimator under the two independent-surveys framework that attains the semiparametric efficiency bound (under a strong-monotonicity identifiability condition) and a robust sub-efficient estimator under a two-stage sampling view that avoids modeling the non-probability mechanism. The authors derive the efficient score, nuisance tangent space, and cross-fitting procedures, and they prove asymptotic normality and efficiency under appropriate conditions; they also provide simulations and a CCTC data application showing when efficiency gains materialize and when robustness is preferable. The methods are implemented in the R package dfSEDI, offering practical guidance for practitioners on when to use the fully efficient vs. sub-efficient approach and how to handle high-dimensional covariates and potential misspecification.

Abstract

Integrating probability and non-probability samples is increasingly important, yet unknown sampling mechanisms in non-probability sources complicate identification and efficient estimation. We develop semiparametric theory for dual-frame data integration and propose two complementary estimators. The first models the non-probability inclusion probability parametrically and attains the semiparametric efficiency bound. We introduce an identifiability condition based on strong monotonicity that identifies sampling-model parameters without instrumental variables, even under informative (non-ignorable) selection, using auxiliary information from the probability sample; it remains valid without record linkage between samples. The second estimator, motivated by a two-stage sampling approximation, avoids explicit modeling of the non-probability mechanism; though not fully efficient, it is efficient within a restricted augmentation class and is robust to misspecification. Simulations and an application to the Culture and Community in a Time of Crisis public simulation dataset show efficiency gains under correct specification and stable performance under misspecification and weak identification. Methods are implemented in the R package \texttt{dfSEDI}.

Figures (2)

• Figure 1: Geometric relationship between the efficient score for $\theta$ (projection onto $\Lambda_2^\perp$), the sub-efficient score (projection onto $\tilde{\Lambda}_2^\perp$), and the information loss incurred by using the restricted projection.
• Figure 2: Boxplots of the estimators for $\phi_3$ and $\theta=E(Y)$ under Scenarios S1--S4. The labels P, NP, P+NP, Eff_S, Eff_union, and Eff represent, respectively: the estimator using only the probability sample; using only the non-probability sample; using units observed in either sample; based on the sub-efficient score; based on the efficient score with setting $h_4^*=\eta_4^*=0$; and with double machine learning DML1 and DML2.

Theorems & Definitions (19)

• Theorem 1: Identifiability
• Remark 1: Positivity condition
• Remark 2: Role of $k(x)$
• Lemma 1
• Theorem 2
• Remark 3: Robustness to missing record-linkage
• Theorem 3: Asymptotic normality and efficiency
• Lemma 2: Projection formulas
• Theorem 4: Efficient and sub-efficient scores
• Remark 4