Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Stochastic Conjugate Subgradient Algorithm for Two-stage Stochastic Programming

Di Zhang, Suvrajeet Sen

TL;DR

The paper tackles two-stage stochastic programming with a non-smooth recourse by proposing the Stochastic Conjugate Subgradient (SCS) framework, which integrates sequential sampling, decomposition, and active-set ideas with Wolfe's non-smooth conjugate subgradient method. The resulting SCS algorithm uses adaptive sampling to form $f_k$, projects subgradients onto the constraint null-space, computes a conjugate search direction via a one-dimensional QP to obtain $\lambda_k^*$, and applies a Wolfe-type line search to guarantee descent. The authors establish convergence and convergence-rate results, including sample-complexity bounds and useful stopping-time estimates, and support theory with preliminary computational results on several two-stage SP datasets showing that SCS yields lower objective values and faster convergence than SGD or stochastic mirror descent. The work advances large-scale SP solving by offering a reliable, scalable method that handles non-smooth recourse and provides actionable convergence guarantees with online data access.

Abstract

Stochastic Optimization is a cornerstone of operations research, providing a framework to solve optimization problems under uncertainty. Despite the development of numerous algorithms to tackle these problems, several persistent challenges remain, including: (i) selecting an appropriate sample size, (ii) determining an effective search direction, and (iii) choosing a proper step size. This paper introduces a comprehensive framework, the Stochastic Conjugate Subgradient (SCS) framework, designed to systematically address these challenges. Specifically, The SCS framework offers structured approaches to determining the sample size, the search direction, and the step size. By integrating various stochastic algorithms within the SCS framework, we have developed a novel stochastic algorithm for two-stage stochastic programming. The convergence and convergence rates of the algorithm have been rigorously established, with preliminary computational results support the theoretical analysis.

A Stochastic Conjugate Subgradient Algorithm for Two-stage Stochastic Programming

TL;DR

The paper tackles two-stage stochastic programming with a non-smooth recourse by proposing the Stochastic Conjugate Subgradient (SCS) framework, which integrates sequential sampling, decomposition, and active-set ideas with Wolfe's non-smooth conjugate subgradient method. The resulting SCS algorithm uses adaptive sampling to form , projects subgradients onto the constraint null-space, computes a conjugate search direction via a one-dimensional QP to obtain , and applies a Wolfe-type line search to guarantee descent. The authors establish convergence and convergence-rate results, including sample-complexity bounds and useful stopping-time estimates, and support theory with preliminary computational results on several two-stage SP datasets showing that SCS yields lower objective values and faster convergence than SGD or stochastic mirror descent. The work advances large-scale SP solving by offering a reliable, scalable method that handles non-smooth recourse and provides actionable convergence guarantees with online data access.

Abstract

Stochastic Optimization is a cornerstone of operations research, providing a framework to solve optimization problems under uncertainty. Despite the development of numerous algorithms to tackle these problems, several persistent challenges remain, including: (i) selecting an appropriate sample size, (ii) determining an effective search direction, and (iii) choosing a proper step size. This paper introduces a comprehensive framework, the Stochastic Conjugate Subgradient (SCS) framework, designed to systematically address these challenges. Specifically, The SCS framework offers structured approaches to determining the sample size, the search direction, and the step size. By integrating various stochastic algorithms within the SCS framework, we have developed a novel stochastic algorithm for two-stage stochastic programming. The convergence and convergence rates of the algorithm have been rigorously established, with preliminary computational results support the theoretical analysis.

Paper Structure

This paper contains 10 sections, 8 theorems, 43 equations, 10 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Connections with the literature
  3. Stochastic Conjugate Subgradient Algorithm
  4. SCS for Two-stage Stochastic Quadratic Programming
  5. Convergence Properties
  6. Sample Complexity Analysis
  7. Feasible direction and sufficient decrease
  8. Convergence Rate and Optimality Condition
  9. Preliminary Computational Results
  10. Conclusion

Key Result

Theorem 1

Given any $0 < \varepsilon <1$, there exists a constant $\kappa > 0$ such that when we have

Figures (10)

  • Figure 1: $\{f(\hat{x}_k)\}_{k=1}^{k=50}$ for different combinations (data,algorithm).
  • Figure 2: $\{f(\hat{x}_k)\}_{k=-50}^{k=-1}$ for different combinations (data,algorithm).
  • Figure : LandS3
  • Figure : lgsc
  • Figure : pgp2
  • ...and 5 more figures

Theorems & Definitions (15)

  • Theorem 1
  • proof
  • Theorem 2
  • proof
  • Theorem 3
  • proof
  • Theorem 4
  • Theorem 5
  • Lemma 6
  • proof
  • ...and 5 more