Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond First-Order Methods for $\ell_p$-Structured Non-Monotone Variational Inequalities

Abhijeet Vyas, Brian Bullins

Abstract

We propose novel high-order algorithms for a class of $\ell_p$-structured non-monotone variational inequalities. In particular, work by Diakonikolas et al. (2021), which introduced the weak Minty variational inequality (weak-MVI) setting, showed how to find an approximate first-order Euclidean stationary point for a strictly positive range of the weak-MVI parameter $ρ$. However, for the $\ell_p$-norm stationary point setting ($p \neq 2$), their guarantees are limited to $ρ=0$, which recovers the standard MVI setting. In this work, we address this gap by presenting a suite of high-order methods that converge to $\ell_p$-norm stationary points for a suitable range of $ρ> 0$, thereby circumventing previous fundamental challenges in $\ell_p$ settings. We further show convergence for high-order smooth \textit{monotone} operators, generalizing Adil et al. (2022) to the case where $p \geq 2$, and we extend our Euclidean techniques to continuous-time settings.

Beyond First-Order Methods for $\ell_p$-Structured Non-Monotone Variational Inequalities

Abstract

We propose novel high-order algorithms for a class of -structured non-monotone variational inequalities. In particular, work by Diakonikolas et al. (2021), which introduced the weak Minty variational inequality (weak-MVI) setting, showed how to find an approximate first-order Euclidean stationary point for a strictly positive range of the weak-MVI parameter . However, for the -norm stationary point setting (), their guarantees are limited to , which recovers the standard MVI setting. In this work, we address this gap by presenting a suite of high-order methods that converge to -norm stationary points for a suitable range of , thereby circumventing previous fundamental challenges in settings. We further show convergence for high-order smooth \textit{monotone} operators, generalizing Adil et al. (2022) to the case where , and we extend our Euclidean techniques to continuous-time settings.
Paper Structure (26 sections, 23 theorems, 162 equations, 3 figures, 3 tables, 3 algorithms)

This paper contains 26 sections, 23 theorems, 162 equations, 3 figures, 3 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Our Contributions
  3. Discrete-time $\ell_p$ setting.
  4. Discrete-time $\ell_2$ setting.
  5. Continuous-time setting.
  6. Additional Related Works
  7. Preliminaries
  8. Discrete Time $\ell_p$-Geometry
  9. Monotone Operators
  10. Weak-MVI
  11. Discrete-Time $\ell_2$-Geometry
  12. Experiments
  13. Continuous-Time $\ell_2$-Geometry
  14. Conclusion
  15. Proofs for $\ell_p$-Geometry
  16. ...and 11 more sections

Key Result

Theorem 3.1

For $s\geq 1,~p \geq 2$, let $F$ be an operator that is $L_{s,p}$$s^{th}$-order smooth and monotone. Upon running the $s^{\text{th}}$ order instance of the $\ell_p$-HOMVI (Algorithm alg:mainalg1) on $F$ we have that the output $z_{out}$ satisfies $\langle F(z), z_{out} - z \rangle \leq \varepsilon$

Figures (3)

  • Figure 1: Visualization of algorithm performance on the modified forsaken example \ref{['example:mforsaken']}.
  • Figure 2: First and second-order methods with $F$ and $F_\alpha$ on (Forsaken). While the algorithm using $F$ cycles, the algorithm using $F_\alpha$ converges to a stationary point. A step size based on a Lipschitz constant of $L_1=10$ and $L_2=500$ is used for the first and second-order methods, respectively.
  • Figure 3: First and second-order methods with $F_{\alpha}$ and $F$. (a) $F = (\nabla_x f,-\nabla_y f)$. (b) $F_{\alpha}:~\alpha = 10$.

Theorems & Definitions (51)

  • Definition 2.1: Stationary points
  • Definition 2.2: Solution set $\mathcal{Z}^*$
  • Definition 2.3: Monotonicity
  • Definition 2.4: $\rho$-comonotone
  • Definition 2.5: Directional derivative
  • Definition 2.6
  • Definition 2.7: $s^{th}$-Order Smoothness in the $\ell_p$-norm
  • Definition 2.8
  • Definition 2.9
  • Definition 2.10
  • ...and 41 more