Decision Making under Model Misspecification: DRO with Robust Bayesian Ambiguity Sets

TL;DR

This work tackles decision-making under model misspecification by combining Distributionally Robust Optimisation with Robust Bayesian Ambiguity Sets (DRO-RoBAS). It replaces KL-based Bayesian ambiguity with an MMD-based, robust Nonparametric Learning (NPL) posterior predictive to form a posterior-informed ambiguity set in RKHS, and derives a dual formulation that enables kernel-based optimisation. Theoretical guarantees show high-probability containment of the true DGP within the RoBAS ball, and empirical results on Newsvendor and Portfolio problems demonstrate improved out-of-sample performance under misspecification, despite higher computational costs. Overall, DRO-RoBAS offers a principled, flexible approach for robust decision-making in the presence of model misspecification and data noise, with potential extensions to other robust Bayesian frameworks and scalable kernel methods.

Abstract

Distributionally Robust Optimisation (DRO) protects risk-averse decision-makers by considering the worst-case risk within an ambiguity set of distributions based on the empirical distribution or a model. To further guard against finite, noisy data, model-based approaches admit Bayesian formulations that propagate uncertainty from the posterior to the decision-making problem. However, when the model is misspecified, the decision maker must stretch the ambiguity set to contain the data-generating process (DGP), leading to overly conservative decisions. We address this challenge by introducing DRO with Robust, to model misspecification, Bayesian Ambiguity Sets (DRO-RoBAS). These are Maximum Mean Discrepancy ambiguity sets centred at a robust posterior predictive distribution that incorporates beliefs about the DGP. We show that the resulting optimisation problem obtains a dual formulation in the Reproducing Kernel Hilbert Space and we give probabilistic guarantees on the tolerance level of the ambiguity set. Our method outperforms other Bayesian and empirical DRO approaches in out-of-sample performance on the Newsvendor and Portfolio problems with various cases of model misspecification.

Figures (5)

• Figure 1: Illustration of (approximated) BASPE, BASPP and RoBAS (ours) with ${\mathbb P}_\theta = {\cal N}(\mu, \sigma^2)$ over a grid of $(\mu, \sigma)$ pairs for a fixed $\epsilon$. In the well-specified case (left), all ambiguity sets include the DGP while RoBAS covers a slightly bigger area than BASPE and BASPP. For a contaminated dataset (right) RoBAS continues to contain the DGP and maintains a similar area, whereas the BAS formulations exclude it and cover a much larger area of distributions further away from the DGP.
• Figure 2: Contaminated Gaussian location example. Top: Histogram of observed data along with the true (DGP), outlier and pathological densities. Bottom: Posterior marginal distributions for NPL-MMD and standard Bayes. The true mean is indicated with a dotted line. For BASPE it holds that ${\mathbb E}_{\Pi_{\text{Bayes}}}[d_{\text{KL}}({\mathbb P}_{\text{pathological}}, {\mathbb P}_\theta)] \approx 0.17<0.42 \approx {\mathbb E}_{\Pi_{\text{Bayes}}}[d_{\text{KL}}({\mathbb P}^\star, {\mathbb P}_\theta)]$ and similarly for BASPP it holds that $d_{\text{KL}}({\mathbb P}_{\text{pathological}}, {\mathbb P}_n^{\text{pred}}) \approx 0.18 < 0.38 \approx d_{\text{KL}}({\mathbb P}^\star, {\mathbb P}_n^{\text{pred}})$. In contrast for RoBAS we have $\mathbb{D}_k({\mathbb P}_{\text{pathological}}, {\mathbb P}_n^{\text{pred(NPL)}}) \approx 0.65 > 0.55 \approx \mathbb{D}_k({\mathbb P}^\star, {\mathbb P}_n^{\text{pred(NPL)}})$. This example is inspired by Fig. 1 of gao2023distributionally.
• Figure 3: The out-of-sample mean and variance for the Newsvendor problem with a misspecified Gaussian location model and a bimodal Gaussian DGP. Results are shown for the univariate ($D=1$, top) and the multivariate ($D=5$, bottom) cases, with markers representing $\epsilon$ values. For illustration purposes, the bottom-left area of the multivariate case is shown in a zoomed-in view.
• Figure 4: Out-of-sample mean-variance trade-off in the Newsvendor problem for a Gaussian location model (top) and an Exponential model (bottom) with a contaminated training dataset. Results are shown for contamination levels $\eta = 0.0$ (left), $\eta = 0.1$ (middle), and $\eta = 0.2$ (right). Each marker represents a specific $\epsilon$ value, with some labelled for reference.
• Figure 5: Out-of-sample mean-variance trade-off in the Portfolio problem for a 5D contaminated Gaussian DGP with $\eta=0.0,0.1,0.2$. Note that the goal is to maximise returns, so larger out-of-sample mean is better.

Theorems & Definitions (20)

• Theorem 3.1: zhu2021kernel, Theorem 3.1
• Proposition 3.2
• Corollary 3.3
• Theorem 3.6
• Remark 3.7
• Corollary 3.8
• Corollary 3.9: Huber's cont. model
• Lemma 1.1
• proof
• proof : Proof of Proposition \ref{['prop:support-fn']}