Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convergence Analysis of Non-Strongly-Monotone Stochastic Quasi-Variational Inequalities

Zeinab Alizadeh, Afrooz Jalilzadeh

TL;DR

This work tackles monotone stochastic quasi-variational inequalities (SQVIs) where the constraint set moves with the decision variable and the operator satisfies a quadratic growth property, a setting that admits non-unique solutions. It develops two inexact algorithms, iEG-SQVI (extra-gradient) and iG-SQVI (gradient), which incorporate an inner solver to handle inexact projections onto the moving constraint set. The authors establish convergence rate results, including linear convergence in outer iterations and an oracle complexity of $\mathcal{O}(1/\epsilon^2)$ in the stochastic case (deterministic case: $\mathcal{O}(\log(1/\epsilon))$), and prove almost sure convergence under Robbins-Siegmund conditions. The methods are demonstrated on an over-parameterized regression game, where bilevel-type lower-level problems are solved with FISTA, illustrating practical effectiveness and robustness in settings with coupling constraints and moving feasibility sets.

Abstract

While Variational Inequality (VI) is a well-established mathematical framework that subsumes Nash equilibrium and saddle-point problems, less is known about its extension, Quasi-Variational Inequalities (QVI). QVI allows for cases where the constraint set changes as the decision variable varies allowing for a more versatile setting. In this paper, we propose extra-gradient and gradient-based methods for solving a class of monotone Stochastic Quasi-Variational Inequalities (SQVI) and establish a rigorous convergence rate analysis for these methods. Our approach not only advances the theoretical understanding of SQVI but also demonstrates its practical applicability. Specifically, we highlight its effectiveness in reformulating and solving problems such as generalized Nash Equilibrium, bilevel optimization, and saddle-point problems with coupling constraints.

Convergence Analysis of Non-Strongly-Monotone Stochastic Quasi-Variational Inequalities

TL;DR

This work tackles monotone stochastic quasi-variational inequalities (SQVIs) where the constraint set moves with the decision variable and the operator satisfies a quadratic growth property, a setting that admits non-unique solutions. It develops two inexact algorithms, iEG-SQVI (extra-gradient) and iG-SQVI (gradient), which incorporate an inner solver to handle inexact projections onto the moving constraint set. The authors establish convergence rate results, including linear convergence in outer iterations and an oracle complexity of in the stochastic case (deterministic case: ), and prove almost sure convergence under Robbins-Siegmund conditions. The methods are demonstrated on an over-parameterized regression game, where bilevel-type lower-level problems are solved with FISTA, illustrating practical effectiveness and robustness in settings with coupling constraints and moving feasibility sets.

Abstract

While Variational Inequality (VI) is a well-established mathematical framework that subsumes Nash equilibrium and saddle-point problems, less is known about its extension, Quasi-Variational Inequalities (QVI). QVI allows for cases where the constraint set changes as the decision variable varies allowing for a more versatile setting. In this paper, we propose extra-gradient and gradient-based methods for solving a class of monotone Stochastic Quasi-Variational Inequalities (SQVI) and establish a rigorous convergence rate analysis for these methods. Our approach not only advances the theoretical understanding of SQVI but also demonstrates its practical applicability. Specifically, we highlight its effectiveness in reformulating and solving problems such as generalized Nash Equilibrium, bilevel optimization, and saddle-point problems with coupling constraints.
Paper Structure (20 sections, 10 theorems, 73 equations, 1 figure, 1 table, 2 algorithms)

This paper contains 20 sections, 10 theorems, 73 equations, 1 figure, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Outline of the Paper
  3. Applications
  4. Generalized Nash Equilibruim
  5. Bilevel Optimization
  6. Saddle Point Problems with Coupling Constraints
  7. Contributions
  8. Preliminaries
  9. Notations
  10. Quadratic Growth Property
  11. Assumptions and Technical Lemmas
  12. Proposed Method
  13. Convergence Analysis
  14. Gradient Approach
  15. Almost Sure Convergence
  16. ...and 5 more sections

Key Result

Lemma 1

bertsekas2003convex Let $X\subseteq \mathbb{R}^n$ be a nonempty closed and convex set. Then the following hold: (a) $\|\mathcal{P}_X [u]- \mathcal{P}_X [v]\| \leq \|u-v\|$ for all $u,v \in \mathbb{R}^n$; (b) $(\mathcal{P}_X [u]-u)^T(x-\mathcal{P} _X [u]) \geq 0$ for all $u \in \mathbb{R}^n$ and $x \

Figures (1)

  • Figure 1: Comparison of iG-SQVI and iEG-SQVI: (Top) for the wikivital mathematics dataset with 300 data points and 1065 features, (Bottom) for a synthetic dataset with 60 data points and 500 features.

Theorems & Definitions (19)

  • Definition 1
  • Lemma 1
  • Definition 2: Gap function.
  • Remark 1
  • Lemma 2
  • proof
  • Definition 3
  • Theorem 1
  • proof
  • Corollary 1: Increasing sample-size
  • ...and 9 more