Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Extragradient Sliding for Composite Non-Monotone Variational Inequalities

Roman Emelyanov, Andrey Tikhomirov, Aleksandr Beznosikov, Alexander Gasnikov

TL;DR

The paper addresses composite variational inequalities of the form $R(x)=Q(x)+P(x)$ where $Q$ is $L_q$-Lipschitz monotone and $P$ is $L_p$-Lipschitz potentially non-monotone under a Minty relaxation. It introduces Extragradient Sliding, a sliding variant of the extragradient method that reduces calls to the expensive non-monotone component and achieves sublinear convergence with a better oracle complexity than classical Extragradient. The main result shows a bound $\min_{0\le j<K-1} \|R(u^j)\|^2 \le \dfrac{16L_p^2 \|x^0 - x^*\|^2}{K}$ and an $P$-oracle complexity of $\mathcal{O}\left( \frac{L_p^2 \|x^0 - x^*\|^2}{\varepsilon^2} \right)$, outperforming $\mathcal{O}\left( \frac{(L_p+L_q)^2 \|x^0 - x^*\|^2}{\varepsilon^2} \right)$ for the basic Extragradient. Numerical experiments on a bilinear saddle point, robust logistic regression, and non-convex least squares corroborate the theoretical gains, showing improved convergence and reduced oracle calls across both monotone and non-monotone components.

Abstract

Variational inequalities offer a versatile and straightforward approach to analyzing a broad range of equilibrium problems in both theoretical and practical fields. In this paper, we consider a composite generally non-monotone variational inequality represented as a sum of $L_q$-Lipschitz monotone and $L_p$-Lipschitz generally non-monotone operators. We applied a special sliding version of the classical Extragradient method to this problem and obtain better convergence results. In particular, to achieve $\varepsilon$-accuracy of the solution, the oracle complexity of the non-monotone operator $Q$ for our algorithm is $O\left(L_p^2/\varepsilon^2\right)$ in contrast to the basic Extragradient algorithm with $O\left((L_p+L_q)^2/\varepsilon^2\right)$. The results of numerical experiments confirm the theoretical findings and show the superiority of the proposed method.

Extragradient Sliding for Composite Non-Monotone Variational Inequalities

TL;DR

The paper addresses composite variational inequalities of the form where is -Lipschitz monotone and is -Lipschitz potentially non-monotone under a Minty relaxation. It introduces Extragradient Sliding, a sliding variant of the extragradient method that reduces calls to the expensive non-monotone component and achieves sublinear convergence with a better oracle complexity than classical Extragradient. The main result shows a bound and an -oracle complexity of , outperforming for the basic Extragradient. Numerical experiments on a bilinear saddle point, robust logistic regression, and non-convex least squares corroborate the theoretical gains, showing improved convergence and reduced oracle calls across both monotone and non-monotone components.

Abstract

Variational inequalities offer a versatile and straightforward approach to analyzing a broad range of equilibrium problems in both theoretical and practical fields. In this paper, we consider a composite generally non-monotone variational inequality represented as a sum of -Lipschitz monotone and -Lipschitz generally non-monotone operators. We applied a special sliding version of the classical Extragradient method to this problem and obtain better convergence results. In particular, to achieve -accuracy of the solution, the oracle complexity of the non-monotone operator for our algorithm is in contrast to the basic Extragradient algorithm with . The results of numerical experiments confirm the theoretical findings and show the superiority of the proposed method.
Paper Structure (10 sections, 3 theorems, 29 equations, 3 figures, 1 algorithm)

This paper contains 10 sections, 3 theorems, 29 equations, 3 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Our contribution.
  3. Notation.
  4. Problem setting
  5. Algorithm
  6. Convergence analysis
  7. Numerical Experiments
  8. Bilinear problem
  9. Logarithmic loss problem
  10. NLLSQ loss problem

Key Result

theorem thmcountertheorem

Consider Algorithm alg1 for the problem eqn::problem under Assumptions asm::R--asm::P, with the following tuning: Assume that $u^k$ (line alg::subproblem) satisfies: Then, we have the following convergence estimate:

Figures (3)

  • Figure 1: Comparison of Extragradient and Extragradient Sliding for the generated bilinear saddle point problem \ref{['eq:bilinear']}.
  • Figure 2: Comparison of Extragradient and Extragradient Sliding for the log loss saddle point problem \ref{['eq:logloss']}
  • Figure 3: Comparison of Extragradient and Extragradient Sliding for the NLLSQ loss saddle point problem \ref{['eq:NLLSQ']}

Theorems & Definitions (4)

  • theorem thmcountertheorem
  • lemma thmcounterlemma
  • corollary thmcountercorollary
  • remark thmcounterremark