Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Statistics of Robust Optimization: A Generalized Empirical Likelihood Approach

John Duchi, Peter Glynn, Hongseok Namkoong

TL;DR

This work develops a generalized empirical likelihood framework for distributionally robust stochastic optimization, providing asymptotically exact confidence intervals for the optimal value and showcasing how robustification acts as a variance regularizer. By extending empirical likelihood to Hadamard differentiable functionals and arbitrary $f$-divergences, the authors obtain pivotal, distribution-free guarantees that apply to both i.i.d. and rapidly mixing dependent data. They connect the robust optimization formulation to risk measures and CVaR-type concepts, and prove consistency and uniform convergence for robust optima under broad conditions. The theory is complemented by simulations on portfolio optimization, CVaR, and multi-item newsvendor problems, illustrating finite-sample performance and calibration. Overall, the paper integrates nonparametric inference with distributionally robust optimization, providing principled tools for uncertainty quantification and robust decision-making in stochastic settings.

Abstract

We study statistical inference and distributionally robust solution methods for stochastic optimization problems, focusing on confidence intervals for optimal values and solutions that achieve exact coverage asymptotically. We develop a generalized empirical likelihood framework---based on distributional uncertainty sets constructed from nonparametric $f$-divergence balls---for Hadamard differentiable functionals, and in particular, stochastic optimization problems. As consequences of this theory, we provide a principled method for choosing the size of distributional uncertainty regions to provide one- and two-sided confidence intervals that achieve exact coverage. We also give an asymptotic expansion for our distributionally robust formulation, showing how robustification regularizes problems by their variance. Finally, we show that optimizers of the distributionally robust formulations we study enjoy (essentially) the same consistency properties as those in classical sample average approximations. Our general approach applies to quickly mixing stationary sequences, including geometrically ergodic Harris recurrent Markov chains.

Statistics of Robust Optimization: A Generalized Empirical Likelihood Approach

TL;DR

This work develops a generalized empirical likelihood framework for distributionally robust stochastic optimization, providing asymptotically exact confidence intervals for the optimal value and showcasing how robustification acts as a variance regularizer. By extending empirical likelihood to Hadamard differentiable functionals and arbitrary -divergences, the authors obtain pivotal, distribution-free guarantees that apply to both i.i.d. and rapidly mixing dependent data. They connect the robust optimization formulation to risk measures and CVaR-type concepts, and prove consistency and uniform convergence for robust optima under broad conditions. The theory is complemented by simulations on portfolio optimization, CVaR, and multi-item newsvendor problems, illustrating finite-sample performance and calibration. Overall, the paper integrates nonparametric inference with distributionally robust optimization, providing principled tools for uncertainty quantification and robust decision-making in stochastic settings.

Abstract

We study statistical inference and distributionally robust solution methods for stochastic optimization problems, focusing on confidence intervals for optimal values and solutions that achieve exact coverage asymptotically. We develop a generalized empirical likelihood framework---based on distributional uncertainty sets constructed from nonparametric -divergence balls---for Hadamard differentiable functionals, and in particular, stochastic optimization problems. As consequences of this theory, we provide a principled method for choosing the size of distributional uncertainty regions to provide one- and two-sided confidence intervals that achieve exact coverage. We also give an asymptotic expansion for our distributionally robust formulation, showing how robustification regularizes problems by their variance. Finally, we show that optimizers of the distributionally robust formulations we study enjoy (essentially) the same consistency properties as those in classical sample average approximations. Our general approach applies to quickly mixing stationary sequences, including geometrically ergodic Harris recurrent Markov chains.

Paper Structure

This paper contains 49 sections, 35 theorems, 169 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Notation
  3. Generalized Empirical Likelihood and Asymptotic Expansions
  4. Generalized Empirical Likelihood for Means
  5. Asymptotic Expansions
  6. Statistical Inference for Stochastic Optimization
  7. Generalized Empirical Likelihood for Stochastic Optimization
  8. Extensions to Dependent Sequences
  9. Computing the Confidence Interval and its Properties
  10. Connections to Robust Optimization and Examples
  11. Upper Confidence Bounds as a Risk Measure
  12. Variance Regularization
  13. Consistency
  14. Uniform Convergence
  15. Consistency for convex problems
  16. ...and 34 more sections

Key Result

Proposition 1

Let Assumption assumption:fdiv hold. Let $Z_i \in \mathbb{R}^d$ be drawn i.i.d. $P_0$ with finite covariance of rank $d_0 \le d$. Then

Figures (1)

  • Figure 1: Average confidence sets for $\inf_{x \in \mathcal{X}} \mathbb{E}_{P_0}[\ell(x; \xi )]$ for normal approximations \ref{['eqn:normal-approximation']} ("Normal") and the generalized empirical likelihood confidence set \ref{['eqn:upper-lower-def']} ("EL"). The true value being approximated in each plot is centered at $0$. The optimal objective computed from the sample average approximation ("SAA") has a negative bias.

Theorems & Definitions (43)

  • Proposition 1
  • Lemma 1
  • Theorem 2
  • Theorem 3
  • Theorem 4
  • Definition 1
  • Theorem 5
  • Proposition 6
  • Lemma 2
  • Lemma 3: Ben-TalHeWaMeRe13
  • ...and 33 more