Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distributionally robust halfspace depth

Jevgenijs Ivanovs, Pavlo Mozharovskyi

Abstract

Tukey's halfspace depth can be seen as a stochastic program and as such it is not guarded against optimizer's curse, so that a limited training sample may easily result in a poor out-of-sample performance. We propose a generalized halfspace depth concept relying on the recent advances in distributionally robust optimization, where every halfspace is examined using the respective worst-case distribution in the Wasserstein ball of radius $δ\geq 0$ centered at the empirical law. This new depth can be seen as a smoothed and regularized classical halfspace depth which is retrieved as $δ\downarrow 0$. It inherits most of the main properties of the latter and, additionally, enjoys various new attractive features such as continuity and strict positivity beyond the convex hull of the support. We provide numerical illustrations of the new depth and its advantages, and develop some fundamental theory. In particular, we study the upper level sets and the median region including their breakdown properties.

Distributionally robust halfspace depth

Abstract

Tukey's halfspace depth can be seen as a stochastic program and as such it is not guarded against optimizer's curse, so that a limited training sample may easily result in a poor out-of-sample performance. We propose a generalized halfspace depth concept relying on the recent advances in distributionally robust optimization, where every halfspace is examined using the respective worst-case distribution in the Wasserstein ball of radius centered at the empirical law. This new depth can be seen as a smoothed and regularized classical halfspace depth which is retrieved as . It inherits most of the main properties of the latter and, additionally, enjoys various new attractive features such as continuity and strict positivity beyond the convex hull of the support. We provide numerical illustrations of the new depth and its advantages, and develop some fundamental theory. In particular, we study the upper level sets and the median region including their breakdown properties.

Paper Structure

This paper contains 30 sections, 25 theorems, 101 equations, 16 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Optimizer's curse
  3. Proposal
  4. Outline
  5. The inner problem
  6. The general solution and the optimal transport plan
  7. The sample version
  8. Properties of the solution
  9. Properties of the robustified halfspace depth
  10. The basic properties
  11. Decay at infinity
  12. Maximin formulation
  13. Finite sample version
  14. Consistency
  15. Breakdown point
  16. ...and 15 more sections

Key Result

Proposition 1

For $\delta\in(0,\infty)$ and ${\boldsymbol z},{\boldsymbol u}\in\mathbb R^d$ with $\| {\boldsymbol u}\|=1$ it holds that where $\lambda=h^{-1}(\delta)\in(0,\infty]$ and eq:sol_alt is not used when $\lambda=\infty$. Furthermore, eq:sol equals 1 iff ${\mathbb E} Y^+\leq \delta$. The minimization problem has the solution: where $\underline\lambda=\underline h^{-1}(\delta)\in(0,\infty]$ and $\under

Figures (16)

  • Figure 1: For normal, skewed normal, and exponential distributions: $320$ bi-variate points (pluses) and ${\boldsymbol z}$ as point (top row), and the corresponding boxplots of the empirical halfspace depth (over $100$ random samples) with point inside a boxplot indicating mean empirical depth (bottom row).
  • Figure 2: Heat maps of the traditional Tukey depth $D_0(\cdot|{\mathbb P}_n)$ (left) and the proposed robust Tukey depth $D_{0.1}(\cdot|{\mathbb P}_n)$ (right) for a sample of $25$ points drawn from a bivariate standard normal distribution. A few contours on the right plot (for depth levels $0.2$, $0.35$, and $0.5$) are presented to emphasize their convexity (later shown formally). With white color being for $0$, the values span from the smallest (shades of yellow) to the highest (shades of red).
  • Figure 3: Depth contours (upper-level sets of the depth function) to the level $\alpha=0.12$ for a sample of $100$ points drawn from a bivariate normal distribution centered at $\boldsymbol 0$ and with the covariance matrix $\bigl((1, 1)^\top, (1, 4)^\top\bigr)$ for the following depth functions: $D_0(\cdot|{\mathbb P})$ (black dashed), $D_0(\cdot|{\mathbb P}_n)$ (black solid), $D_\delta(\cdot|{\mathbb P}_n)$ (red), and $\underline D_\delta(\cdot|{\mathbb P}_n)$ (blue). We use $\delta=0.05$ (left), $\delta=0.015$ (middle), and $\delta=0.005$ (right).
  • Figure 4: Illustration of the solution in \ref{['eq:sol_emp']}: $p=1/n$ and $k=3$ with the result in $(3/n,4/n]$. The optimal transport plan \ref{['eq:opt_transport']} is depicted by blue arrows, where the dashed arrow indicates that less than $1/n$ mass is moved.
  • Figure 5: $\overline\alpha(\delta)$ and ${\alpha}^*(\delta)$ (red) for the standard normal in an arbitrary dimension.
  • ...and 11 more figures

Theorems & Definitions (52)

  • Proposition 1: Inner problem
  • Corollary 1: Sample version
  • Lemma 1: Monotonicity
  • Lemma 2: Mixture
  • Lemma 3: Boundary values
  • Lemma 4: Strict monotonicity
  • Lemma 5: Continuity
  • Proposition 2: Standard properties
  • Proposition 3: Additional properties
  • Proposition 4: Tukey depth approximation
  • ...and 42 more