Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stochastic variance reduced extragradient methods for solving hierarchical variational inequalities

Pavel Dvurechensky, Andrea Ebner, Johannes Carl Schnebel, Shimrit Shtern, Mathias Staudigl

TL;DR

This work is the first to prove convergence rates and complexity statements for variance-reduced stochastic algorithms approaching the solution of hierarchical VIs in Euclidean and Bregman setups.

Abstract

We are concerned with optimization in a broad sense through the lens of solving variational inequalities (VIs) -- a class of problems that are so general that they cover as particular cases minimization of functions, saddle-point (minimax) problems, Nash equilibrium problems, and many others. The key challenges in our problem formulation are the two-level hierarchical structure and finite-sum representation of the smooth operators in each level. For this setting, we are the first to prove convergence rates and complexity statements for variance-reduced stochastic algorithms approaching the solution of hierarchical VIs in Euclidean and Bregman setups.

Stochastic variance reduced extragradient methods for solving hierarchical variational inequalities

TL;DR

This work is the first to prove convergence rates and complexity statements for variance-reduced stochastic algorithms approaching the solution of hierarchical VIs in Euclidean and Bregman setups.

Abstract

We are concerned with optimization in a broad sense through the lens of solving variational inequalities (VIs) -- a class of problems that are so general that they cover as particular cases minimization of functions, saddle-point (minimax) problems, Nash equilibrium problems, and many others. The key challenges in our problem formulation are the two-level hierarchical structure and finite-sum representation of the smooth operators in each level. For this setting, we are the first to prove convergence rates and complexity statements for variance-reduced stochastic algorithms approaching the solution of hierarchical VIs in Euclidean and Bregman setups.
Paper Structure (31 sections, 15 theorems, 152 equations, 2 figures, 2 algorithms)

This paper contains 31 sections, 15 theorems, 152 equations, 2 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related works
  3. Variational inequalities
  4. Hierarchical VIs
  5. Outline of results and comparison
  6. Euclidean Setup
  7. Preliminaries
  8. Gap functions for hierarchical VI's.
  9. Hierarchical extragradient with variance reduction
  10. Analysis
  11. Bregman Setup
  12. Analysis
  13. Numerical Experiments
  14. Equilibrium Selection.
  15. Linearly Constrained Equilibrium.
  16. ...and 16 more sections

Key Result

Lemma 2.3

Consider problem eq:P. Let Assumption ass:standing and ass:CQ hold. Let $\mathcal{U}_1\subseteq\mathop{\mathrm{dom}}\nolimits(g_{1})$ be a nonempty compact set with $\mathcal{U}_1\cap\mathcal{S}_{1}\neq\varnothing$. Then, there exists a constant $B_{\mathcal{U}_1}>0$ such that Suppose $\mathcal{S}_{2}$ is $(\kappa,\rho)$-weakly sharp. Then for all nonempty and compact subsets $\mathcal{U}_2\subse

Figures (2)

  • Figure 1: Performance in the Equilibrium Selection problem
  • Figure 2: Performance in the Linearly Constrained Equilibrium problem

Theorems & Definitions (28)

  • Example 1.1: equilibrium selection
  • Example 1.2: Hierarchical games
  • Remark 2.1
  • Example 2.1
  • Definition 2.1
  • Definition 2.2: Weak Sharpness
  • Lemma 2.3
  • Remark 2.2
  • Lemma 2.4
  • Lemma 2.5
  • ...and 18 more