Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adjusted Wasserstein Distributionally Robust Estimator in Statistical Learning

Yiling Xie, Xiaoming Huo

TL;DR

This work advances distributionally robust statistical learning by correcting the asymptotic bias of the classic WDRO estimator through a nonlinear transformation, yielding an adjusted estimator (ADRO) with asymptotic unbiasedness and reduced mean-squared error. The authors develop a general adjustment framework based on a sequential delta method, prove its validity, and instantiate it within generalized linear models, deriving explicit ADRO forms for logistic, Poisson, and linear regression. They show that ADRO preserves out-of-sample guarantees while improving asymptotic efficiency, and they confirm practical benefits with numerical experiments demonstrating superior finite-sample performance. The approach is easy to implement given the WDRO solution and holds promise for broader deployment in robust inference under Wasserstein ambiguity.

Abstract

We propose an adjusted Wasserstein distributionally robust estimator -- based on a nonlinear transformation of the Wasserstein distributionally robust (WDRO) estimator in statistical learning. The classic WDRO estimator is asymptotically biased, while our adjusted WDRO estimator is asymptotically unbiased, resulting in a smaller asymptotic mean squared error. Further, under certain conditions, our proposed adjustment technique provides a general principle to de-bias asymptotically biased estimators. Specifically, we will investigate how the adjusted WDRO estimator is developed in the generalized linear model, including logistic regression, linear regression, and Poisson regression. Numerical experiments demonstrate the favorable practical performance of the adjusted estimator over the classic one.

Adjusted Wasserstein Distributionally Robust Estimator in Statistical Learning

TL;DR

This work advances distributionally robust statistical learning by correcting the asymptotic bias of the classic WDRO estimator through a nonlinear transformation, yielding an adjusted estimator (ADRO) with asymptotic unbiasedness and reduced mean-squared error. The authors develop a general adjustment framework based on a sequential delta method, prove its validity, and instantiate it within generalized linear models, deriving explicit ADRO forms for logistic, Poisson, and linear regression. They show that ADRO preserves out-of-sample guarantees while improving asymptotic efficiency, and they confirm practical benefits with numerical experiments demonstrating superior finite-sample performance. The approach is easy to implement given the WDRO solution and holds promise for broader deployment in robust inference under Wasserstein ambiguity.

Abstract

We propose an adjusted Wasserstein distributionally robust estimator -- based on a nonlinear transformation of the Wasserstein distributionally robust (WDRO) estimator in statistical learning. The classic WDRO estimator is asymptotically biased, while our adjusted WDRO estimator is asymptotically unbiased, resulting in a smaller asymptotic mean squared error. Further, under certain conditions, our proposed adjustment technique provides a general principle to de-bias asymptotically biased estimators. Specifically, we will investigate how the adjusted WDRO estimator is developed in the generalized linear model, including logistic regression, linear regression, and Poisson regression. Numerical experiments demonstrate the favorable practical performance of the adjusted estimator over the classic one.
Paper Structure (41 sections, 19 theorems, 151 equations, 9 figures)

This paper contains 41 sections, 19 theorems, 151 equations, 9 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Organization of this Paper
  4. Adjustment Technique
  5. Sequential Delta Method
  6. WDRO Problem
  7. Problem Formulation
  8. Asymptotic Distribution of the WDRO Estimator
  9. Proposed Adjusted WDRO Estimator
  10. Definition and Existence
  11. Simplification of the Adjusted WDRO Estimator
  12. Asymptotically Unbiased
  13. Out-of-sample Performance Guarantee
  14. Adjusted WDRO in the Generalized Linear Model
  15. Formulation of the Generalized Linear Model
  16. ...and 26 more sections

Key Result

Proposition 1

Suppose $\beta_n$ is an estimator of ground-truth parameter $\beta_\ast$ and has the following convergence in distribution: where $f$ is differentiable at some neighborhood $\mathcal{B}(\beta_\ast)$ of $\beta_\ast$. Assume the transformation $\phi_n$ is differentiable at $\mathcal{B}(\beta_\ast)$ and satisfies $\phi_n(\beta)\to \phi(\beta)$ and $\phi_n^{\prime}(\beta)\to \phi^{\prime}(\beta)$ for

Figures (9)

  • Figure 1: Histogram of $\beta_n^{DRO}$
  • Figure 2: Squared error and log loss plots of the logistic regression, $\tau=1.5$.
  • Figure 3: Squared error and log loss plots of the logistic regression, $\tau=2$.
  • Figure 4: Squared error and log loss plots of the logistic regression, $\tau=2.5$.
  • Figure 5: Squared error and log loss plots of the logistic regression, $\tau=3$.
  • ...and 4 more figures

Theorems & Definitions (24)

  • Proposition 1
  • Theorem 3: Adjustement Technique
  • Theorem 4: Sequential Delta Method
  • Remark 6
  • Theorem 9: Extension of Theorem 1 in blanchet2022confidence
  • Remark 10
  • Remark 11: Finite Sample Size
  • Definition 12: Adjusted WDRO Estimator
  • Proposition 13: Existence of Adjusted WDRO Estimator I
  • Proposition 14: Existence of Adjusted WDRO Estimator II
  • ...and 14 more