Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Learning-based Surrogate Method for Stochastic Programs with Implicitly Decision-dependent Uncertainty

Boyang Shen, Junyi Liu

TL;DR

This work tackles stochastic programs with decision-dependent uncertainty by modeling the latent randomness nonparametrically and developing an Adaptive Learning-based Surrogate (ALS) method that jointly learns surrogates, performs simulation, and optimizes. It proves that the SP-DDU objective is $\tau$-weakly convex and offers a framework for near-stationarity in expectation via the Moreau envelope, enabling nonasymptotic NSE guarantees under variable proximal parameters and mini-batch sizes. The ALS algorithm integrates adaptive/ static simulation oracles with learning-based surrogates and proximal updates, providing rigorous convergence and complexity results, including faster rates when sample sizes grow and proximal parameters increase judiciously. Numerical experiments on production/pricing, facility location, and spam classification demonstrate improved stability, convergence speed, and robustness against distributional shifts compared to PO-based and stochastic approximation approaches. The framework highlights practical potential for decision-dependent uncertainty in real-world settings and points to extensions to multistage and chance-constrained problems.

Abstract

We consider a class of stochastic programming problems where the implicitly decision-dependent random variable follows a nonparametric regression model with heteroscedastic error. The Clarke subdifferential and surrogate functions are not readily obtainable due to the latent decision dependency. To deal with such a computational difficulty, we develop an adaptive learning-based surrogate method that integrates the simulation scheme and statistical estimates to construct estimation-based surrogate functions in a way that the simulation process is adaptively guided by the algorithmic procedure. We establish the non-asymptotic convergence rate analysis in terms of $(ν, δ)$-near stationarity in expectation under variable proximal parameters and batch sizes, which exhibits the superior convergence performance and enhanced stability in both theory and practice. We provide numerical results with both synthetic and real data which illustrate the benefits of the proposed algorithm in terms of algorithmic stability and efficiency.

Adaptive Learning-based Surrogate Method for Stochastic Programs with Implicitly Decision-dependent Uncertainty

TL;DR

This work tackles stochastic programs with decision-dependent uncertainty by modeling the latent randomness nonparametrically and developing an Adaptive Learning-based Surrogate (ALS) method that jointly learns surrogates, performs simulation, and optimizes. It proves that the SP-DDU objective is -weakly convex and offers a framework for near-stationarity in expectation via the Moreau envelope, enabling nonasymptotic NSE guarantees under variable proximal parameters and mini-batch sizes. The ALS algorithm integrates adaptive/ static simulation oracles with learning-based surrogates and proximal updates, providing rigorous convergence and complexity results, including faster rates when sample sizes grow and proximal parameters increase judiciously. Numerical experiments on production/pricing, facility location, and spam classification demonstrate improved stability, convergence speed, and robustness against distributional shifts compared to PO-based and stochastic approximation approaches. The framework highlights practical potential for decision-dependent uncertainty in real-world settings and points to extensions to multistage and chance-constrained problems.

Abstract

We consider a class of stochastic programming problems where the implicitly decision-dependent random variable follows a nonparametric regression model with heteroscedastic error. The Clarke subdifferential and surrogate functions are not readily obtainable due to the latent decision dependency. To deal with such a computational difficulty, we develop an adaptive learning-based surrogate method that integrates the simulation scheme and statistical estimates to construct estimation-based surrogate functions in a way that the simulation process is adaptively guided by the algorithmic procedure. We establish the non-asymptotic convergence rate analysis in terms of -near stationarity in expectation under variable proximal parameters and batch sizes, which exhibits the superior convergence performance and enhanced stability in both theory and practice. We provide numerical results with both synthetic and real data which illustrate the benefits of the proposed algorithm in terms of algorithmic stability and efficiency.
Paper Structure (36 sections, 13 theorems, 98 equations, 4 figures, 4 tables, 1 algorithm)

This paper contains 36 sections, 13 theorems, 98 equations, 4 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminary analysis on weak convexity and stationarity
  3. Surrogate models for SP-DDU Problem
  4. Conceptual surrogate functions
  5. Learning-based surrogate functions
  6. Adaptive simulation oracle
  7. Static simulation oracle
  8. Learning-based surrogate functions
  9. Estimation error analysis
  10. ALS algorithm and nonasymptotic convergence analysis
  11. ALS algorithm with variable parameters
  12. Convergence analysis of the ALS algorithm
  13. Complexity of the ALS algorithm
  14. Complexity with constant sample sizes
  15. Constant proximal parameter
  16. ...and 21 more sections

Key Result

Proposition 1

For a lower semicontinuous function $f: \mathcal{O} \to \mathbb R\cup\{+\infty\}$ and $\rho >0$, the following statements are equivalent:

Figures (4)

  • Figure 1: Comparison of the PO paradigm with the integrated paradigm.
  • Figure 2: Performance of the ALS algorithm under different settings of proximal parameters and mini-batch sampling schemes.
  • Figure 3: Performance of the ALS algorithm under the three simulation oracles.
  • Figure 4: Numerical performance for the facility location problem.

Theorems & Definitions (18)

  • Definition 1: weakly convex function
  • Proposition 1: characterization of weak convexity
  • Proposition 2
  • Definition 2: near stationarity in expectation
  • Proposition 3: Moreau envelope and near stationarity
  • Proposition 4
  • proof
  • Theorem 1
  • Proposition 5
  • Proposition 6
  • ...and 8 more