Robust Mean Estimation With Auxiliary Samples

TL;DR

The paper tackles robust mean estimation when auxiliary data is available within a Wasserstein-2 neighborhood of the target distribution. It develops a minimax framework with a linear estimator and reduces the worst-case risk to Gaussian first- and second-moment statistics, yielding closed-form optimal weights $\\mathbf A = s\\mathbf I$, $\\mathbf B =\\mathbf I -\\mathbf A$ and explicit minimax risks $R_F^*, R_{Tr}^*, R_{op}^*$. The key result is that auxiliary data improves MSE notably when the Wasserstein radius is small relative to the target variance and the number of true samples is limited, with precise expressions showing how $n,N,d,\\delta,\\epsilon$ interact. Numerical experiments in a Gaussian location model corroborate the theory, illustrating when data augmentation is beneficial and how the optimal weighting behaves as $\\epsilon$ varies. Overall, the work provides fundamental limits and practical estimators for robust mean estimation with auxiliary samples under distributional shift modeled by $\\W_2$-balls.

Abstract

In data-driven learning and inference tasks, the high cost of acquiring samples from the target distribution often limits performance. A common strategy to mitigate this challenge is to augment the limited target samples with data from a more accessible "auxiliary" distribution. This paper establishes fundamental limits of this approach by analyzing the improvement in the mean square error (MSE) when estimating the mean of the target distribution. Using the Wasserstein-2 metric to quantify the distance between distributions, we derive expressions for the worst-case MSE when samples are drawn (with labels) from both a target distribution and an auxiliary distribution within a specified Wasserstein-2 distance from the target distribution. We explicitly characterize the achievable MSE and the optimal estimator in terms of the problem dimension, the number of samples from the target and auxiliary distributions, the Wasserstein-2 distance, and the covariance of the target distribution. We note that utilizing samples from the auxiliary distribution effectively improves the MSE when the squared radius of the Wasserstein-2 uncertainty ball is small compared to the variance of the true distribution and the number of samples from the true distribution is limited. Numerical simulations in the Gaussian location model illustrate the theoretical findings.

Figures (1)

• Figure 1: MSE of optimal estimator from Theorem \ref{['thm: main']}.

Theorems & Definitions (7)

• Definition 1
• Lemma 1
• proof
• Theorem 1
• Theorem 2
• Theorem 3
• proof