Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Methods for Variational Inequalities under Relaxed Smoothness Assumption

Daniil Vankov, Angelia Nedich, Lalitha Sankar

TL;DR

This work proves the first-known convergence results for solving generalized smooth VIs for the three popular methods, namely, projection, Korpelevich, and Popov methods, and presents numerical experiments that support the theoretical guarantees and highlight the efficiency of proposed adaptive stepsizes.

Abstract

Variational Inequality (VI) problems have attracted great interest in the machine learning (ML) community due to their application in adversarial and multi-agent training. Despite its relevance in ML, the oft-used strong-monotonicity and Lipschitz continuity assumptions on VI problems are restrictive and do not hold in practice. To address this, we relax smoothness and monotonicity assumptions and study structured non-monotone generalized smoothness. The key idea of our results is in adaptive stepsizes. We prove the first-known convergence results for solving generalized smooth VIs for the three popular methods, namely, projection, Korpelevich, and Popov methods. Our convergence rate results for generalized smooth VIs match or improve existing results on smooth VIs. We present numerical experiments that support our theoretical guarantees and highlight the efficiency of proposed adaptive stepsizes.

Adaptive Methods for Variational Inequalities under Relaxed Smoothness Assumption

TL;DR

This work proves the first-known convergence results for solving generalized smooth VIs for the three popular methods, namely, projection, Korpelevich, and Popov methods, and presents numerical experiments that support the theoretical guarantees and highlight the efficiency of proposed adaptive stepsizes.

Abstract

Variational Inequality (VI) problems have attracted great interest in the machine learning (ML) community due to their application in adversarial and multi-agent training. Despite its relevance in ML, the oft-used strong-monotonicity and Lipschitz continuity assumptions on VI problems are restrictive and do not hold in practice. To address this, we relax smoothness and monotonicity assumptions and study structured non-monotone generalized smoothness. The key idea of our results is in adaptive stepsizes. We prove the first-known convergence results for solving generalized smooth VIs for the three popular methods, namely, projection, Korpelevich, and Popov methods. Our convergence rate results for generalized smooth VIs match or improve existing results on smooth VIs. We present numerical experiments that support our theoretical guarantees and highlight the efficiency of proposed adaptive stepsizes.
Paper Structure (15 sections, 17 theorems, 247 equations, 2 figures, 1 table)

This paper contains 15 sections, 17 theorems, 247 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Assumptions on VI
  3. Methods with Adaptive Stepsizes
  4. Convergence Analysis for Projection Method
  5. Convergence Analysis for Korpelevich Method
  6. Convergence Analysis for Popov method
  7. Adaptive Clipping
  8. Numerical Results
  9. Concluding Remarks
  10. Proof of Proposition \ref{['example1']}
  11. Technical Lemmas
  12. Projection Method Analysis
  13. Korpelevich Method Analysis
  14. Popov Method Analysis
  15. Analysis of Korpelevich method with Adaptive Clipping \ref{['Adaptive-Clipping']}

Key Result

Proposition 2.2

Let $U\subseteq \mathbb{R}^m$ be a nonempty set and let $F(\cdot): U\to\mathbb{R}^m$ be an operator. Then, we have

Figures (2)

  • Figure 1: Comparison of projection, Korpelevich and Popov methods and Korpelevich method with golden ratio stepsizes for different $(\alpha, p)$.
  • Figure 2: Comparison of projection, Korpelevich and Popov methods and Korpelevich method with golden ratio stepsizes for different $(\alpha, p)$.

Theorems & Definitions (27)

  • Proposition 2.2: DBLP:conf/icml/00020LL23, Proposition 1
  • Proposition 2.5
  • Theorem 4.1
  • Corollary 4.2
  • Lemma 5.1
  • Theorem 5.2
  • Theorem 5.3
  • Lemma 6.1
  • Theorem 6.2
  • Theorem 6.3
  • ...and 17 more