Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Minimum Wasserstein distance estimator under covariate shift: closed-form, super-efficiency and irregularity

Junjun Lang, Qiong Zhang, Yukun Liu

TL;DR

This work addresses parameter estimation under covariate shift and MAR by introducing the minimum Wasserstein distance (W) estimator, which avoids explicit outcome modeling or density-weight estimation. The W-estimator has a closed-form, is equivalent to 1-NN, and admits root-$n$ asymptotic normality, with potential super-efficiency relative to semiparametric bounds under certain conditions, despite not being asymptotically linear. The authors extend the framework to missing data via MAR, derive enhanced W-estimation using regression augmentation, and demonstrate strong finite-sample performance through simulations and a rainfall data analysis. The approach offers a principled, geometry-aware alternative to weighting and plug-in methods, with practical robustness and the flexibility to incorporate modern regression tools while maintaining valid inference.

Abstract

Covariate shift arises when covariate distributions differ between source and target populations while the conditional distribution of the response remains invariant, and it underlies problems in missing data and causal inference. We propose a minimum Wasserstein distance estimation framework for inference under covariate shift that avoids explicit modeling of outcome regressions or importance weights. The resulting W-estimator admits a closed-form expression and is numerically equivalent to the classical 1-nearest neighbor estimator, yielding a new optimal transport interpretation of nearest neighbor methods. We establish root-$n$ asymptotic normality and show that the estimator is not asymptotically linear, leading to super-efficiency relative to the semiparametric efficient estimator under covariate shift in certain regimes, and uniformly in missing data problems. Numerical simulations, along with an analysis of a rainfall dataset, underscore the exceptional performance of our W-estimator.

Minimum Wasserstein distance estimator under covariate shift: closed-form, super-efficiency and irregularity

TL;DR

This work addresses parameter estimation under covariate shift and MAR by introducing the minimum Wasserstein distance (W) estimator, which avoids explicit outcome modeling or density-weight estimation. The W-estimator has a closed-form, is equivalent to 1-NN, and admits root- asymptotic normality, with potential super-efficiency relative to semiparametric bounds under certain conditions, despite not being asymptotically linear. The authors extend the framework to missing data via MAR, derive enhanced W-estimation using regression augmentation, and demonstrate strong finite-sample performance through simulations and a rainfall data analysis. The approach offers a principled, geometry-aware alternative to weighting and plug-in methods, with practical robustness and the flexibility to incorporate modern regression tools while maintaining valid inference.

Abstract

Covariate shift arises when covariate distributions differ between source and target populations while the conditional distribution of the response remains invariant, and it underlies problems in missing data and causal inference. We propose a minimum Wasserstein distance estimation framework for inference under covariate shift that avoids explicit modeling of outcome regressions or importance weights. The resulting W-estimator admits a closed-form expression and is numerically equivalent to the classical 1-nearest neighbor estimator, yielding a new optimal transport interpretation of nearest neighbor methods. We establish root- asymptotic normality and show that the estimator is not asymptotically linear, leading to super-efficiency relative to the semiparametric efficient estimator under covariate shift in certain regimes, and uniformly in missing data problems. Numerical simulations, along with an analysis of a rainfall dataset, underscore the exceptional performance of our W-estimator.
Paper Structure (14 sections, 26 equations, 5 figures, 1 table, 2 algorithms)

This paper contains 14 sections, 26 equations, 5 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. W-estimator under covariate shift
  3. Minimum Wasserstein distance estimator
  4. Asymptotic properties of the W-estimator
  5. Enhanced W-estimation
  6. Extension to missing data problems
  7. Minimum Wasserstein distance estimator
  8. Super-efficiency and enhanced W-estimation
  9. Simulation studies
  10. Empirical verification of theoretical results
  11. Covariate shift
  12. Missing data
  13. Real data analysis
  14. Discussion

Figures (5)

  • Figure 1: Empirical verification of Theorem \ref{['thm-asy-theta-hat-TL']}.
  • Figure 2: Empirical verification of Proposition \ref{['prop-var-est']}.
  • Figure 3: MSE of various estimators under regression functions (i)--(vi) from left to right, top to bottom when XGBoost is used for regression. Within each panel, results are shown for different combinations of source sample size $n$ and target sample size $m$.
  • Figure 4: MSE of various estimators under regression functions (i)--(vi) from left to right, top to bottom when linear model is used for regression. Within each panel, results are shown for different combinations of source sample size $n$ and target sample size $m$.
  • Figure 5: Boxplots of the SE $(\hat{\theta}_r - \theta^*)^2$ for various estimators over $R=500$ repetitions under the MAR setting, across different sample sizes, with both the outcome regression and propensity score models correctly specified.