Minimax Statistical Estimation under Wasserstein Contamination

TL;DR

This work develops a minimax theory for estimation under Wasserstein-$r$ contaminations in the $\ell_q$ setting, introducing joint (JC) and independent (IC) perturbations and analyzing location estimation, linear regression, and density estimation. Using optimal transport tools such as the dynamic Benamou–Brenier formulation, it proves exact minimax risks for JC location estimation (with the sample mean being minimax) and sharp bounds for IC, including exact Euclidean results and Gaussian special cases. For linear regression, the least-squares predictor and estimator are shown to be near or exactly minimax optimal under JC/IC perturbations, with precise bounds that depend on design conditioning; a phase-transition phenomenon arises in the IC $W_{2,2}$ setting. In nonparametric density estimation, the minimax rate under Wasserstein contamination is the maximum of the classical rate and a perturbation term, achievable by kernel density estimation with a perhaps enlarged bandwidth. Overall, the results reveal that classical estimators can be nearly robust to Wasserstein-type contamination in many regimes, while highlighting regimes (e.g., ill-conditioned designs) where robustness remains challenging and new methods may be needed.

Abstract

Contaminations are a key concern in modern statistical learning, as small but systematic perturbations of all datapoints can substantially alter estimation results. Here, we study Wasserstein-$r$ contaminations ($r\ge 1$) in an $\ell_q$ norm ($q\in [1,\infty]$), in which each observation may undergo an adversarial perturbation with bounded cost, complementing the classical Huber model, corresponding to total variation norm, where only a fraction of observations is arbitrarily corrupted. We study both independent and joint (coordinated) contaminations and develop a minimax theory under $\ell_q^r$ losses. Our analysis encompasses several fundamental problems: location estimation, linear regression, and pointwise nonparametric density estimation. For joint contaminations in location estimation and for prediction in linear regression, we obtain the exact minimax risk, identify least favorable contaminations, and show that the sample mean and least squares predictor are respectively minimax optimal. For location estimation under independent contaminations, we give sharp upper and lower bounds, including exact minimaxity in the Euclidean Wasserstein contamination case, when $q=r=2$. For pointwise density estimation in any dimension, we derive the optimal rate, showing that it is achieved by kernel density estimation with a bandwidth that is possibly larger than the classical one. Our proofs leverage powerful tools from optimal transport developed over the last 20 years, including the dynamic Benamou-Brenier formulation. Taken together, our results suggest that in contrast to the Huber contamination model, for norm-based Wasserstein contaminations, classical estimators may be nearly optimally robust.

Figures (3)

• Figure 1: Representation of IC/JC adversarial perturbations for location estimation with $W_{2,2}$ perturbations and $\ell_2^2$ loss, for $n=3$, as per \ref{['lf']}. We plot the true $\theta$ (blue), the unobserved clean observations (black), the observed perturbed observations (red), as well as the corresponding sample means. The second plot uses a small value of $\varepsilon$ where $\zeta>0$ and $\psi=0$, while the third plot uses a larger value of $\varepsilon$ where $\zeta>0$ and $\psi>0$.
• Figure 2: Left: Values of $\zeta$ and $\psi$ for IC perturbations for varying $\varepsilon$, as per \ref{['eq: zeta psi']}. We see that for small values of $\varepsilon$, $\zeta$ increases while $\psi$ is zero, and past the transition (gray) of $\varepsilon=\sigma/(n-1)$, $\zeta$ remains constant at $1/(n-1)$ while $\psi$ increases with $\varepsilon$. Right: For the IC Gaussian mean estimation from Section \ref{['euw2']}, we plot the log squared error divided by $\log n$, for $n=10$, $p=3$, and $\Sigma=6^{-1}\mathrm{diag}(1,2,3)$ (so that $\mathop{\mathrm{Tr}}\nolimits(\Sigma)=1$), and the IC transition represented with the gray dashed line. For small values of $\varepsilon$, we see a rate of $n^{-1}$, as expected.
• Figure 3: Left: Graphical representation of linear regression with an adversarial perturbation with the least squares estimate $P_X Y$ of $X\theta$. Here, $Y'$ is constructed so that $Y'-Y$ is a scaled multiple of $P_X Y-X\theta$. The length is proportional to the perturbation budget, and is indicated for the case $r=q=2$ on the figure. Right: Empirical evaluation of the risk in a linear regression problem with $r=q=2$ under homoskedastic error and prediction loss with $n=10$, $p=5$. The empirical performance matches the theoretical rate.

Theorems & Definitions (49)

• Definition 1.1: Contaminations
• Definition 1.2: Minimax risk and estimators
• Theorem 2.2: Location parameter estimation under JC
• Theorem 2.3: Location family IC
• Theorem 2.4: IC Location Estimation, $r = q = 2$
• proof : Proof of Theorem \ref{['thm: gaussian mean opt']}.
• Corollary 2.5: IC location under $W_{2,1}$ for spherical Gaussian noise
• Theorem 3.2: Linear regression under JC
• Lemma 3.3: Scaling $+$ signed shift along $\mathsf{C}$
• proof