Semiparametric semi-supervised learning for general targets under distribution shift and decaying overlap

TL;DR

The paper addresses statistical inference for semi-supervised learning when labels are missing at random (MAR) and overlap decays with sample size, which invalidates classical root-$n$ inference. It develops a general semiparametric framework (DS3) that constructs doubly robust, asymptotically normal estimators for general targets under decaying MAR-SS, using influence-function theory and allowing distribution shift between labeled and unlabeled data. A key contribution is the nonstandard rate governed by the effective sample size $(n+N)a_{n,N}$, which captures overlap decay, and the extension to prediction-powered inference (PPI) under MAR by incorporating $f(X)$ into the propensity structure. The framework is validated through simulations for multivariate means and linear coefficients and demonstrated on real data from BLE-RSSI localization and METABRIC breast cancer, illustrating practical gains in inference when labels are scarce and biased.

Abstract

In modern scientific applications, large volumes of covariate data are readily available, while outcome labels are costly, sparse, and often subject to distribution shift. This asymmetry has spurred interest in semi-supervised (SS) learning, but most existing approaches rely on strong assumptions -- such as missing completely at random (MCAR) labeling or strict positivity -- that put substantial limitations on their practical usefulness. In this work, we introduce a general semiparametric framework for estimation and inference in SS settings where labels are missing at random (MAR) and the overlap may vanish as sample size increases. Our framework accommodates a wide range of smooth statistical targets -- including means, linear coefficients, quantiles, and causal effects -- and remains valid under high-dimensional nuisance estimation and distributional shift between labeled and unlabeled samples. We construct estimators that are doubly robust and asymptotically normal by deriving influence functions under this decaying MAR-SS regime. A key insight is that classical root-$n$ convergence fails under vanishing overlap; we instead provide corrected asymptotic rates that capture the impact of the decay in overlap. We validate our theory through simulations and demonstrate practical utility in real-world applications on the internet of things and breast cancer where labeled data are scarce.

Figures (22)

• Figure 1: Multivariate outcome mean simulation results. The simulation is run $100$ times under the decaying offset scenario with $n = 100$ labeled samples and $N = 1000$ unlabeled samples; the outcome model $\hat{\mu}$ is estimated using linear regression, and propensity score model $\hat{\pi}$ is estimated via logistic regression with offset. The left panel displays boxplots summarizing the RMSE distribution over different seeds of the considered estimators. The right panel shows the empirical coverage and $\pm2$ standard error bars around them.
• Figure 2: Linear regression coefficients simulation results. The simulation is run $100$ times under the decaying offset scenario with $n = 100$ labeled samples and $N = 1000$ unlabeled samples; the outcome model $\hat{\mu}$ is estimated using linear regression, and propensity score model $\hat{\pi}$ is estimated via logistic regression with offset. The left panel displays boxplots summarizing the RMSE distribution over different seeds of the considered estimators. The right panel shows the empirical coverage and $\pm2$ standard error bars around them.
• Figure 3: BLE-RSSI application results. Left panel: estimated propensity scores across observations, along with the proportion of labeled data $n / (n + N)$, shown as a vertical dashed line. The heterogeneity in the estimated propensity scores may suggest a missing-at-random (MAR) labeling mechanism. Right panel: joint bivariate distribution over the floor bivariate grid of the observed position of the device $Y$ compared to the distribution of predicted positions from the estimated outcome model using signal recorded by remote sensors. The discrepancy between observed and predicted outcomes may suggest a missing-at-random (MAR) labeling mechanism. Dots represent bivariate means computed using our approach and the naive approach.
• Figure 4: METABRIC application results. Left panel: estimated propensity scores across observations, along with the proportion of labeled data $n / (n + N)$, shown as a vertical dashed line. The heterogeneity in the estimated propensity scores may suggest a missing-at-random (MAR) labeling mechanism. Right panel: marginal distribution of the observed survival times $Y$ (in months) compared to the distribution of predicted survival times from the estimated outcome model using both biomarkers and clinical covariates. Vertical lines indicate the means of each distribution. The discrepancy between observed and predicted outcomes further suggests a MAR mechanism. Bottom panel: estimated linear coefficients from our semi-supervised approach and the naive estimator based solely on labeled data. Confidence intervals are at $\alpha=0.1$ significance level.
• Figure C.1: Multivariate outcome mean simulation results. Empirical coverage is reported for the setting with logistic true propensity score $\pi^\star$, linear outcome regression model $\hat{\mu}$, and a constant propensity model $\hat{\pi}$. Each panel corresponds to a specific configuration of labeled and unlabeled sample sizes, with $n \in \{100, 1000, 10000\}$ and $N \in \{10n, 100n\}$.

Theorems & Definitions (19)

• Remark 2.2
• Remark 2.3: Triangular arrays
• Proposition 3.1: Observed-data influence function
• Remark 3.3
• Remark 3.4: Lindeberg condition
• Theorem 3.5: Consistency and asymptotic normality, known $\pi^\star$
• Remark 3.6: Effective sample size
• Theorem 3.7: Consistency and asymptotic normality, unknown nuisances
• Remark 3.8: Asymptotic normality under misspecification
• Corollary 3.9: PPI asymptotic normality under decaying MAR-SS