Sample Average Approximation for Stochastic Programming with Equality Constraints

TL;DR

The paper addresses the challenge that naively applying SAA to stochastic programs with equality constraints can fail to guarantee asymptotic feasibility or optimality. It proposes relaxing equality constraints with a shrinking tolerance δ_N and proves, under mild smoothness and boundedness assumptions on the constraint functions, that the SAA approximations are asymptotically optimal, with convergence analyzed through random compact sets and concentration inequalities. A core technical contribution is establishing convergence of random level sets and providing Hölder-based concentration bounds that relax Lipschitz requirements. The framework is applied to stochastic optimal control under Wiener disturbances, including a Mars rocket descent example, demonstrating both theoretical consistency and practical variance reduction in final-state outcomes compared to a deterministic baseline.

Abstract

We revisit the sample average approximation (SAA) approach for non-convex stochastic programming. We show that applying the SAA approach to problems with expected value equality constraints does not necessarily result in asymptotic optimality guarantees as the sample size increases. To address this issue, we relax the equality constraints. Then, we prove the asymptotic optimality of the modified SAA approach under mild smoothness and boundedness conditions on the equality constraint functions. Our analysis uses random set theory and concentration inequalities to characterize the approximation error from the sampling procedure. We apply our approach and analysis to the problem of stochastic optimal control for nonlinear dynamical systems under external disturbances modeled by a Wiener process. Numerical results on relevant stochastic programs show the reliability of the proposed approach. Results on a rocket-powered descent problem show that our computed solutions allow for significant uncertainty reduction compared to a deterministic baseline.

Figures (6)

• Figure 1: Two infeasible examples for the standard SAA approach.
• Figure 1: $\mathbb{D}$-distance between the solution sets $S_0^\star$ and $S^\star_N(\bar{\omega})$ to P and to $\textbf{SP}\xspace_N(\bar{\omega})$, respectively. If $\mathbb{D}(S^\star_N(\bar{\omega}),S_0^\star)\to 0$ as $N\to\infty$ as in Theorem \ref{['thm:main:rand_sets']}, then the limit of the solutions sets $S^\star_N(\bar{\omega})$ is a subset of the solution set $S_0^\star$. Thus, in the limit as the sample size $N$ increases, solutions $u_N^\star(\bar{\omega})\in S^\star_N(\bar{\omega})$ to $\textbf{SP}\xspace_N(\bar{\omega})$ are optimal solutions to P.
• Figure 1: Example that does not satify \ref{['A4b']}: there exists no $u\in\mathcal{U}$ that satisfies $g_0(u)<0$ and $\|u-u^\star\|\leq 1$.
• Figure 1: Benchmark study. Mean success rates (representing successful numerical resolutions of the sampled problems) and median optimal value errors for different choices of $\delta_N$ and $N$.
• Figure 1: Conditions of Theorem \ref{['thm:conv_randSets_detLim']}.

Theorems & Definitions (33)

• Theorem 2.1: Asymptotic optimality of the SAA approach for problems with expectation equality constraints
• Theorem 3.1
• Lemma 3.2: Almost-sure convergence of random level sets
• Remark 3.3
• Corollary 3.4
• Proposition 3.5: Concentration for $\alpha$-Hölder continuous function classes
• Proposition 3.6: Concentration for $\alpha$-Hölder bounded function classes
• Corollary 3.7
• Proof 1: Proof of Theorem \ref{['thm:main:h_lipschitz_bounded']} (asymptotic optimality of the SAA approach)
• Theorem 4.1: Asymptotic optimality of the SAA approach