Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distributionally Robust Optimisation with Bayesian Ambiguity Sets

Charita Dellaporta, Patrick O'Hara, Theodoros Damoulas

TL;DR

Distributionally Robust Optimisation with Bayesian Ambiguity Sets (DRO-BAS) is introduced which hedges against uncertainty in the model by optimising the worst-case risk over a posterior-informed ambiguity set.

Abstract

Decision making under uncertainty is challenging since the data-generating process (DGP) is often unknown. Bayesian inference proceeds by estimating the DGP through posterior beliefs about the model's parameters. However, minimising the expected risk under these posterior beliefs can lead to sub-optimal decisions due to model uncertainty or limited, noisy observations. To address this, we introduce Distributionally Robust Optimisation with Bayesian Ambiguity Sets (DRO-BAS) which hedges against uncertainty in the model by optimising the worst-case risk over a posterior-informed ambiguity set. We show that our method admits a closed-form dual representation for many exponential family members and showcase its improved out-of-sample robustness against existing Bayesian DRO methodology in the Newsvendor problem.

Distributionally Robust Optimisation with Bayesian Ambiguity Sets

TL;DR

Distributionally Robust Optimisation with Bayesian Ambiguity Sets (DRO-BAS) is introduced which hedges against uncertainty in the model by optimising the worst-case risk over a posterior-informed ambiguity set.

Abstract

Decision making under uncertainty is challenging since the data-generating process (DGP) is often unknown. Bayesian inference proceeds by estimating the DGP through posterior beliefs about the model's parameters. However, minimising the expected risk under these posterior beliefs can lead to sub-optimal decisions due to model uncertainty or limited, noisy observations. To address this, we introduce Distributionally Robust Optimisation with Bayesian Ambiguity Sets (DRO-BAS) which hedges against uncertainty in the model by optimising the worst-case risk over a posterior-informed ambiguity set. We show that our method admits a closed-form dual representation for many exponential family members and showcase its improved out-of-sample robustness against existing Bayesian DRO methodology in the Newsvendor problem.
Paper Structure (26 sections, 10 theorems, 33 equations, 3 figures, 3 tables)

This paper contains 26 sections, 10 theorems, 33 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. DRO with Bayesian Ambiguity Sets
  3. The Newsvendor Problem
  4. Experiment setup.
  5. Analysis.
  6. Discussion
  7. Special cases
  8. Gaussian model with unknown mean and known variance
  9. Tolerance level $\epsilon$
  10. Gaussian Model with Unknown Mean and Variance
  11. Tolerance level $\epsilon$
  12. Exponential likelihood with conjugate gamma prior
  13. Tolerance level $\epsilon$
  14. Proofs of Theoretical Results
  15. Proofs of DRO-BAS upper bound in Equation (\ref{['eq:kl-upper-bound']})
  16. ...and 11 more sections

Key Result

Lemma 1

Let $p(\xi \mid \theta)$ be an exponential family likelihood and $\pi(\theta)$, $\Pi(\theta \mid {\cal D})$ a conjugate prior-posterior pair, also members of the exponential family. Let $\tau_0, \tau_n \in T$ be hyperparameters of the prior and posterior respectively, where $T$ is the hyperparameter then the expected KL-divergence can be written as:

Figures (3)

  • Figure 1: Illustration of the construction of the BDRO and DRO-BAS optimisation problems for three posterior samples $\theta_1, \theta_2, \theta_3 \mathop{\mathrm{\overset{iid}{\sim}}}\nolimits \Pi(\theta \mid {\cal D})$. BDRO seeks the decision that minimises the average worst-case risk between the three ambiguity sets shown in figure (a) whereas DRO-BAS targets the decision minimising the worst-case risk over the ambiguity set shown in (b).
  • Figure 2: The out-of-sample mean-variance tradeoff (in bold) while varying $\epsilon$ for DRO-BAS and BDRO when the total number of samples from the model is 25 (left), 100 (middle), and 900 (right).
  • Figure 3: The out-of-sample mean-variance tradeoff on the truncated-normal dataset when varying the radius $\epsilon$ for DRO-BAS and BDRO when the total number of samples from the model is 25 (left), 100 (middle), and 900 (right).

Theorems & Definitions (24)

  • Lemma 1
  • Theorem 1
  • Definition 1: Gaussian with unknown mean and known variance
  • Lemma 2
  • proof
  • Corollary 1
  • proof
  • Definition 2: Unknown mean and variance of a Gaussian
  • Definition 3
  • Lemma 3
  • ...and 14 more