Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Solutions of Two-stage Stochastic Minimax Problems

Hailin Sun, Xiaojun Chen

TL;DR

This work tackles a two-stage stochastic minimax problem where the first-stage objective is nonconvex-concave and the second-stage problem is strongly convex-concave. It develops a rigorous analysis of the second-stage value functions, saddle/minimax/KKT relationships, and the interchangeability between expectation and minimax operators, establishing existence and regularity results. Using a sample average approximation, the authors prove almost-sure convergence of the KKT point sets as the sample size grows and propose an Inexact Parallel Proximal Gradient Descent Ascent (IPPGDA) algorithm, augmented with a semi-smooth Newton method for inner solves, with subsequence and global convergence guarantees via KL and semialgebraic arguments. Numerical experiments on a two-stage stochastic two-player zero-sum game validate the approach and illustrate convergence behavior and SAA properties, highlighting the method’s robustness to stochasticity and scalability to larger samples.

Abstract

This paper introduces a class of two-stage stochastic minimax problems where the first-stage objective function is nonconvex-concave while the second-stage objective function is strongly convex-concave. We establish properties of the second-stage minimax value function and solution functions, and characterize the existence and relationships among saddle points, minimax points, and KKT points. We apply the sample average approximation (SAA) to the class of two-stage stochastic minimax problems and prove the convergence of the KKT points as the sample size tends to infinity. An inexact parallel proximal gradient descent ascent algorithm is proposed to solve this class of problems with the SAA. Numerical experiments demonstrate the effectiveness of the proposed algorithm and validate the convergence properties of the SAA approach.

Solutions of Two-stage Stochastic Minimax Problems

TL;DR

This work tackles a two-stage stochastic minimax problem where the first-stage objective is nonconvex-concave and the second-stage problem is strongly convex-concave. It develops a rigorous analysis of the second-stage value functions, saddle/minimax/KKT relationships, and the interchangeability between expectation and minimax operators, establishing existence and regularity results. Using a sample average approximation, the authors prove almost-sure convergence of the KKT point sets as the sample size grows and propose an Inexact Parallel Proximal Gradient Descent Ascent (IPPGDA) algorithm, augmented with a semi-smooth Newton method for inner solves, with subsequence and global convergence guarantees via KL and semialgebraic arguments. Numerical experiments on a two-stage stochastic two-player zero-sum game validate the approach and illustrate convergence behavior and SAA properties, highlighting the method’s robustness to stochasticity and scalability to larger samples.

Abstract

This paper introduces a class of two-stage stochastic minimax problems where the first-stage objective function is nonconvex-concave while the second-stage objective function is strongly convex-concave. We establish properties of the second-stage minimax value function and solution functions, and characterize the existence and relationships among saddle points, minimax points, and KKT points. We apply the sample average approximation (SAA) to the class of two-stage stochastic minimax problems and prove the convergence of the KKT points as the sample size tends to infinity. An inexact parallel proximal gradient descent ascent algorithm is proposed to solve this class of problems with the SAA. Numerical experiments demonstrate the effectiveness of the proposed algorithm and validate the convergence properties of the SAA approach.

Paper Structure

This paper contains 14 sections, 20 theorems, 136 equations, 2 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Examples, assumptions and concepts
  3. Properties of two-stage stochastic minimax problems
  4. Interchange of the expectation and the minimax operator
  5. Global and local minimax points
  6. Sample average approximation of the two-stage stochastic minimax problem
  7. The inexact parallel proximal gradient descent ascent algorithm for two-stage stochastic minimax problem
  8. The key estimates of IPPGDA algorithm
  9. Perturbed gradient-like descent sequence
  10. Subsequence and global convergence
  11. A semi-smooth Newton method for problem \ref{['eq:nonsmooth-equation']}
  12. Numerical experiments
  13. Conclusion
  14. Appendix: A generalization of Theorem 2G.8 in dontchev2009implicit (implicit function theorem for stationary points)

Key Result

Lemma 3.1

Under Assumptions a:second-stage-a:xicompact-regular, the following statements hold.

Figures (2)

  • Figure 1: Residual values versus iteration numbers with different starting points and random matrices for $\tau=0.1$ and $0.5$, respectively
  • Figure 2: Convergence of the SAA problems when $X_1 = [-10, 10]^{3}$ and $X_1= [-20, 20]^{3}$, respectively

Theorems & Definitions (59)

  • Example 2.1
  • Example 2.2
  • Remark 2.1
  • Definition 2.1
  • Definition 2.2
  • Definition 2.3
  • Remark 2.2
  • Definition 2.4
  • Definition 2.5
  • Lemma 3.1
  • ...and 49 more