Multi-cut stochastic approximation methods for solving stochastic convex composite optimization

TL;DR

This work develops a multi-cut stochastic approximation framework for stochastic convex composite optimization (SCCO), defining cutting-plane models as the pointwise maxima of stochastic linearizations to solve $\phi_* = \min_x \{ f(x) + h(x) \}$ with $f(x) = \mathbb{E}_{\xi}[F(x,\xi)]$. It introduces the stochastic cutting plane (S-CP) framework and instantiates a max-one-cut method (S-Max1C), proving a near-optimal $\tilde{O}(1/\sqrt{I})$ convergence rate, with a multistage warm-start variant (MS-Max1C) that preserves the same rate and offers ${\cal O}(N I)$ iteration complexity. The paper provides rigorous analysis of the noise in the max-based cuts, including bounds on ${\cal N}(u;\Gamma)$ and recursive control via auxiliary quantities, and demonstrates practical performance gains over RSA, DA, and SCPB on two two-stage stochastic programs. Numerical experiments show that S-Max1C and MS-Max1C achieve competitive or superior objective values across diverse instances while maintaining reasonable runtimes, highlighting the value of max-based multi-cut approaches for SCCO. Overall, the work opens a new avenue for scalable SCCO algorithms with robust theoretical guarantees and favorable empirical behavior, with potential extensions such as smoothing the max and exploring alternative bundle constructions.

Abstract

The development of a multi-cut stochastic approximation (SA) method for solving stochastic convex composite optimization (SCCO) problems has remained an open challenge. The difficulty arises from the fact that the stochastic multi-cut model, constructed as the pointwise maximum of individual stochastic linearizations, provides a biased estimate of the objective function, with the error being uncontrollable. This paper introduces multi-cut SA methods for solving SCCO problems, achieving near-optimal convergence rates. The cutting-plane models used in these methods are the pointwise maxima of appropriately chosen one-cut models. To the best of our knowledge, these are the first multi-cut SA methods specifically designed for SCCO problems. Finally, computational experiments demonstrate that these methods generally outperform both the robust stochastic approximation method and the stochastic dual averaging method across all instances tested.

Theorems & Definitions (25)

• Proposition 2.1
• Theorem 2.2
• proof
• Theorem 3.1
• Lemma 3.2
• proof
• Lemma 3.3
• proof
• Proposition 3.4
• proof