Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimal Analysis of Method with Batching for Monotone Stochastic Finite-Sum Variational Inequalities

Alexander Pichugin, Maksim Pechin, Aleksandr Beznosikov, Alexander Gasnikov

TL;DR

This paper presents an analysis of a method that gives optimal convergence estimates for monotone stochastic finite-sum variational inequalities and supports batching and does not lose the oracle complexity optimality.

Abstract

Variational inequalities are a universal optimization paradigm that is interesting in itself, but also incorporates classical minimization and saddle point problems. Modern realities encourage to consider stochastic formulations of optimization problems. In this paper, we present an analysis of a method that gives optimal convergence estimates for monotone stochastic finite-sum variational inequalities. In contrast to the previous works, our method supports batching and does not lose the oracle complexity optimality. The effectiveness of the algorithm, especially in the case of small but not single batches is confirmed experimentally.

Optimal Analysis of Method with Batching for Monotone Stochastic Finite-Sum Variational Inequalities

TL;DR

This paper presents an analysis of a method that gives optimal convergence estimates for monotone stochastic finite-sum variational inequalities and supports batching and does not lose the oracle complexity optimality.

Abstract

Variational inequalities are a universal optimization paradigm that is interesting in itself, but also incorporates classical minimization and saddle point problems. Modern realities encourage to consider stochastic formulations of optimization problems. In this paper, we present an analysis of a method that gives optimal convergence estimates for monotone stochastic finite-sum variational inequalities. In contrast to the previous works, our method supports batching and does not lose the oracle complexity optimality. The effectiveness of the algorithm, especially in the case of small but not single batches is confirmed experimentally.
Paper Structure (6 sections, 5 theorems, 48 equations, 2 figures, 1 table, 1 algorithm)

This paper contains 6 sections, 5 theorems, 48 equations, 2 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Setup and Assumptions
  3. Main Part
  4. Experiments
  5. Conclusion
  6. Proof of Theorem \ref{['th:ALg1']}

Key Result

theorem 1

Consider the problem eq:VI+eq:sum under Assumptions ass. Let $\{x^k\}$ be the sequence generated by Algorithm alg:sarah with tuning of $\eta, \theta, \alpha, \beta, \gamma$ as follows: Then, given $\varepsilon>0$, the number of iterations for $\mathbb E[\text{Gap}(x^k)] \leq \varepsilon$ is

Figures (2)

  • Figure 1: Comparison of computational complexities for Algorithm \ref{['alg:sarah']}, EG-Mal22-1 (Algorithm 1 alacaoglu2021stochastic), EG-Mal22-2 (Algorithm 2 alacaoglu2021stochastic), EG-Car19 (Algorithm 1+2 from carmon2019variance) on \ref{['bilinear']} with policeman and burglar matrix from nemirovski2013mini.
  • Figure 2: Comparison of computational complexities for Algorithm \ref{['alg:sarah']}, EG-Mal22-1 (Algorithm 1 alacaoglu2021stochastic), EG-Mal22-2 (Algorithm 2 alacaoglu2021stochastic), EG-Car19 (Algorithm 1+2 from carmon2019variance) on \ref{['bilinear']} with policeman and burglar matrix from nemirovski2013mini.

Theorems & Definitions (7)

  • theorem 1
  • corollary 1
  • lemma 1
  • lemma 2
  • proof
  • lemma 3
  • proof