Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Solving Stochastic Variational Inequalities without the Bounded Variance Assumption

Ahmet Alacaoglu, Jun-Hyun Kim

TL;DR

The paper addresses stochastic variational inequalities (SVIs) with unbounded domains and without bounded-variance assumptions, focusing on monotone VIs and structured nonmonotone VIs admitting a weak Minty VI solution. It develops and analyzes three first-order approaches—minibatch forward–backward–forward (FBF), MLMC-based inexact fixed-point FBF, and a variance-reduced Halpern-anchored FBF—to achieve a residual accuracy of $\mathbb{E}[\mathrm{res}(\mathbf{z})]\le \varepsilon$ with a near-optimal stochastic oracle complexity of $\widetilde{O}(\varepsilon^{-4})$ under progressively weaker variance conditions and larger permissible nonmonotonicity (via bounds on $\rho$). The methods do not rely on bounded variance or bounded domains and extend to constrained min–max problems through the regularizer $r$, with numerical results demonstrating stability and robustness beyond the traditional extragradient framework. The work advances the understanding of SVIs in realistic, unbounded settings and offers practical, single- or near-single-loop algorithms with strong theoretical guarantees.

Abstract

We analyze algorithms for solving stochastic variational inequalities (VI) without the bounded variance or bounded domain assumptions, where our main focus is min-max optimization with possibly unbounded constraint sets. We focus on two classes of problems: monotone VIs; and structured nonmonotone VIs that admit a solution to the weak Minty VI. The latter assumption allows us to solve structured nonconvex-nonconcave min-max problems. For both classes of VIs, to make the expected residual norm less than $\varepsilon$, we show an oracle complexity of $\widetilde{O}(\varepsilon^{-4})$, which is the best-known for constrained VIs. In our setting, this complexity had been obtained with the bounded variance assumption in the literature, which is not even satisfied for bilinear min-max problems with an unbounded domain. We obtain this complexity for stochastic oracles whose variance can grow as fast as the squared norm of the optimization variable.

Solving Stochastic Variational Inequalities without the Bounded Variance Assumption

TL;DR

The paper addresses stochastic variational inequalities (SVIs) with unbounded domains and without bounded-variance assumptions, focusing on monotone VIs and structured nonmonotone VIs admitting a weak Minty VI solution. It develops and analyzes three first-order approaches—minibatch forward–backward–forward (FBF), MLMC-based inexact fixed-point FBF, and a variance-reduced Halpern-anchored FBF—to achieve a residual accuracy of with a near-optimal stochastic oracle complexity of under progressively weaker variance conditions and larger permissible nonmonotonicity (via bounds on ). The methods do not rely on bounded variance or bounded domains and extend to constrained min–max problems through the regularizer , with numerical results demonstrating stability and robustness beyond the traditional extragradient framework. The work advances the understanding of SVIs in realistic, unbounded settings and offers practical, single- or near-single-loop algorithms with strong theoretical guarantees.

Abstract

We analyze algorithms for solving stochastic variational inequalities (VI) without the bounded variance or bounded domain assumptions, where our main focus is min-max optimization with possibly unbounded constraint sets. We focus on two classes of problems: monotone VIs; and structured nonmonotone VIs that admit a solution to the weak Minty VI. The latter assumption allows us to solve structured nonconvex-nonconcave min-max problems. For both classes of VIs, to make the expected residual norm less than , we show an oracle complexity of , which is the best-known for constrained VIs. In our setting, this complexity had been obtained with the bounded variance assumption in the literature, which is not even satisfied for bilinear min-max problems with an unbounded domain. We obtain this complexity for stochastic oracles whose variance can grow as fast as the squared norm of the optimization variable.
Paper Structure (27 sections, 14 theorems, 153 equations, 3 figures, 1 table, 3 algorithms)

This paper contains 27 sections, 14 theorems, 153 equations, 3 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Assumptions
  3. Contributions and Comparisons
  4. Discussion about \ref{['tab:constraints']}.
  5. Results with Minibatching
  6. Algorithmic Ideas
  7. Complexity Analysis
  8. Results with MLMC
  9. Algorithmic Ideas
  10. Complexity Analysis
  11. Variance Reduction
  12. Algorithmic Ideas
  13. Complexity Analysis
  14. Proof sketch.
  15. Numerical Results
  16. ...and 12 more sections

Key Result

Theorem 2.1

Let Assumptions asp: 1 and asp: 3 hold and suppose that $\rho < \frac{1}{12L}$. For the algorithm in eq: fbf_stoc with gradient estimators computed as eq: fbf_mb with $b_k = \Theta(k\log(k+1))$ and $\eta_k = \Theta(1/L)$ (where precise parameters are given in Thm. th: mb_supp), we have that where $\mathbf{z}^{\text{out}}$ is generated by selecting an index $\hat{k}$ uniformly at random after runn

Figures (3)

  • Figure 1: Left: Alg. \ref{['alg:weakmvi_stoc']} and EG for the counter-example problem (see \ref{['eq: def_rot']}). Middle and right: Alg. \ref{['alg: var_red_fbf']} and the algorithm of pethick2023solving for the unconstrained problem in \ref{['eq: uq_prob']} with operator noise distributed as Student's t and Laplace-distribution.
  • Figure 2: Left: Trajectories of Alg.1 and EG for the LAx counter-example (see \ref{['eq: def_rot']}). Middle: Alg.1 for counter-example with varying $\rho$. Right: Methods from kotsalis2022simpleii, iusem2017extragradient, alacaoglu2025towards in \ref{['tab:constraints']} for the LAx counter-example. Middle and Right panel has log-scaled $y$-axis.
  • Figure 3: Alg. 4 and the algorithm of pethick2023solving for the unconstrained problem in \ref{['eq: uq_prob']} with Gaussian noise. Panel has log-scaled $y$-axis.

Theorems & Definitions (28)

  • Theorem 2.1
  • Remark 2.2
  • Theorem 3.1
  • Remark 3.2
  • Lemma 3.3
  • Lemma 3.4
  • Theorem 4.1
  • Remark 4.2
  • proof
  • Lemma B.1
  • ...and 18 more