Robust Mean Estimation for Optimization: The Impact of Heavy Tails

TL;DR

This work addresses safe mean estimation for heavy-tailed losses within data-driven optimization by employing KL-DRO. It proves that a KL-divergence-based ambiguity set yields a least-conservative estimator with robust, exponential-overestimation guarantees, even when data are regularly varying with tail index $\rho>1$. The paper derives sharp non-asymptotic and asymptotic bounds (including a uniform over distributions bound) and establishes the optimality of KL-DRO among all estimators sharing the same exponential disappointment rate. It also demonstrates that KL-DRO can outperform common alternatives (Wasserstein DRO, truncation, and variance regularization) in heavy-tailed regimes, and it does so without requiring bounded support. These results extend KL-DRO’s favorable statistical properties to heavy tails and provide principled guidance for robust decision-making under-tail risks in practice.

Abstract

We consider the problem of constructing a least conservative estimator of the expected value $μ$ of a non-negative heavy-tailed random variable. We require that the probability of overestimating the expected value $μ$ is kept appropriately small; a natural requirement if its subsequent use in a decision process is anticipated. In this setting, we show it is optimal to estimate $μ$ by solving a distributionally robust optimization (DRO) problem using the Kullback-Leibler (KL) divergence. We further show that the statistical properties of KL-DRO compare favorably with other estimators based on truncation, variance regularization, or Wasserstein DRO.

Theorems & Definitions (30)

• Proposition 2.1
• Proposition 2.2
• Proposition 2.3
• Theorem 3.1
• Proposition 3.2
• Theorem 3.3
• Lemma 4.1
• proof : Proof of Proposition \ref{['prop-klub']}
• proof : Proof of Lemma \ref{['lemma:upper-bound-log-log-nonasymptotic']}.
• Proposition 5.1