Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Solutions of Stochastic Optimization Problems with Bayesian Neural Networks

Alan A. Lahoud, Erik Schaffernicht, Johannes A. Stork

TL;DR

This work addresses decision-making under uncertainty in data-driven optimization by modeling unknown OP parameters with Bayesian Neural Networks and propagating the resulting predictive distribution through stochastic optimization. It introduces two learning paradigms: Decoupled learning, which fits the parameter distribution and then solves a stochastic OP via MC sampling, and Combined learning, which differentiates through the end-to-end solver to directly minimize the downstream cost. The framework is evaluated on Newsvendor, its quadratic variant, and a portfolio risk problem using synthetic and real data, demonstrating reduced decision regret $R$ and free-aleatoric regret $FR$ relative to strong baselines, with the Combined approach often delivering the best performance under limited data or sampling. The results provide guidance on when to apply each method and highlight the practical impact of sampling choices and data availability for robust data-driven optimization.

Abstract

Mathematical solvers use parametrized Optimization Problems (OPs) as inputs to yield optimal decisions. In many real-world settings, some of these parameters are unknown or uncertain. Recent research focuses on predicting the value of these unknown parameters using available contextual features, aiming to decrease decision regret by adopting end-to-end learning approaches. However, these approaches disregard prediction uncertainty and therefore make the mathematical solver susceptible to provide erroneous decisions in case of low-confidence predictions. We propose a novel framework that models prediction uncertainty with Bayesian Neural Networks (BNNs) and propagates this uncertainty into the mathematical solver with a Stochastic Programming technique. The differentiable nature of BNNs and differentiable mathematical solvers allow for two different learning approaches: In the Decoupled learning approach, we update the BNN weights to increase the quality of the predictions' distribution of the OP parameters, while in the Combined learning approach, we update the weights aiming to directly minimize the expected OP's cost function in a stochastic end-to-end fashion. We do an extensive evaluation using synthetic data with various noise properties and a real dataset, showing that decisions regret are generally lower (better) with both proposed methods.

Learning Solutions of Stochastic Optimization Problems with Bayesian Neural Networks

TL;DR

This work addresses decision-making under uncertainty in data-driven optimization by modeling unknown OP parameters with Bayesian Neural Networks and propagating the resulting predictive distribution through stochastic optimization. It introduces two learning paradigms: Decoupled learning, which fits the parameter distribution and then solves a stochastic OP via MC sampling, and Combined learning, which differentiates through the end-to-end solver to directly minimize the downstream cost. The framework is evaluated on Newsvendor, its quadratic variant, and a portfolio risk problem using synthetic and real data, demonstrating reduced decision regret and free-aleatoric regret relative to strong baselines, with the Combined approach often delivering the best performance under limited data or sampling. The results provide guidance on when to apply each method and highlight the practical impact of sampling choices and data availability for robust data-driven optimization.

Abstract

Mathematical solvers use parametrized Optimization Problems (OPs) as inputs to yield optimal decisions. In many real-world settings, some of these parameters are unknown or uncertain. Recent research focuses on predicting the value of these unknown parameters using available contextual features, aiming to decrease decision regret by adopting end-to-end learning approaches. However, these approaches disregard prediction uncertainty and therefore make the mathematical solver susceptible to provide erroneous decisions in case of low-confidence predictions. We propose a novel framework that models prediction uncertainty with Bayesian Neural Networks (BNNs) and propagates this uncertainty into the mathematical solver with a Stochastic Programming technique. The differentiable nature of BNNs and differentiable mathematical solvers allow for two different learning approaches: In the Decoupled learning approach, we update the BNN weights to increase the quality of the predictions' distribution of the OP parameters, while in the Combined learning approach, we update the weights aiming to directly minimize the expected OP's cost function in a stochastic end-to-end fashion. We do an extensive evaluation using synthetic data with various noise properties and a real dataset, showing that decisions regret are generally lower (better) with both proposed methods.
Paper Structure (36 sections, 9 equations, 3 figures, 2 tables)

This paper contains 36 sections, 9 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Problem Formulation
  4. OPs under uncertainty.
  5. Dataset and Problem Statement.
  6. Probabilistic Model as Bayesian Neural Networks
  7. Methods
  8. Decoupled Learning with BNN
  9. Combined Learning with BNN
  10. Evaluation
  11. Experiments
  12. Classical Newsvendor Problem
  13. Data and OP parameters.
  14. Quadratic Programming Newsvendor
  15. Data and OP parameters.
  16. ...and 21 more sections

Figures (3)

  • Figure 1: A diagram depicting the proposed addition of a BNN distribution predictor block before solving data-driven OPs with a mathematical solver in a stochastic fashion. The solid lines illustrate the decision inference given new input data, while the dotted lines indicate the learning process, where the BNN weight updates can be computed based on prediction quality (Decoupled learning, 1) or decision quality (Combined learning, 2).
  • Figure 2: Variation of $FR$ with sampling size and training data.
  • Figure 3: For the NV1 OP, the C-ANN, D-BNN, and D-BNN methods achieve good decisions with different strategies for the OP parameters predictions.