A Lyapunov analysis of Korpelevich's extragradient method with fast and flexible extensions

TL;DR

The paper addresses solving monotone inclusions of the form $0 \in F(z) + \partial g(z)$ by introducing a Lyapunov-based analysis for Korpelevich’s extragradient method, yielding a last-iterate $o(1/k)$ convergence rate for a unifying Lyapunov function that upper-bounds common optimality measures. It then designs three FLEX-type algorithms that blend standard extragradient steps with user-specified directions under Lyapunov-guided line-search, guaranteeing global convergence and enabling superlinear rates when directions are well-chosen (e.g., quasi-Newton, Anderson acceleration). Theoretical results establish descent, quasi-Fejér properties, and convergence for the three variants (FLEX, I-FLEX, Prox-FLEX), with additional results under strong monotonicity and injectivity. Numerical experiments across quadratic minimax problems, bilinear games, Cournot–Nash equilibria, and sparse logistic regression demonstrate the practical effectiveness and simplicity of the approach, particularly when leveraging accelerated directions like Anderson or J-symmetric updates. Overall, the work broadens extragradient analysis to include Lyapunov-based globalization and flexible direction strategies, offering practical globalization tools for fast local directions in monotone inclusion problems.

Abstract

We develop a Lyapunov-based analysis of Korpelevich's extragradient method and show that it achieves an $o(1/k)$ last-iterate convergence rate of the constructed Lyapunov function. This Lyapunov function simultaneously upper bounds several standard measures of optimality, which allows our analysis to sharpen existing last-iterate convergence guarantees for these measures. Moreover, the same analysis enables the design of a class of flexible extensions of the extragradient method in which extragradient steps are adaptively blended with user-specified directions via a Lyapunov-guided line-search procedure. These extensions retain global convergence under practical assumptions and can attain superlinear rates when the directions are chosen appropriately. Numerical experiments confirm the simplicity and efficiency of the proposed framework.

Figures (4)

• Figure 1: Convergence of algorithms on the quadratic minimax problem \ref{['eq:quadratic_minimax']}. Both AA-I and AA-II use memory parameter $m=20$. When $\omega = 0$, the operator $F$ in \ref{['eq:quadratic_minimax:F']} is monotone; for $\omega > 0$, it becomes strongly monotone.
• Figure 2: Convergence of algorithms on the bilinear zero-sum game with simplex constraints \ref{['eq:bilinear']} where $r^{k} = R_{1/2L_F}(z^k)$ and $R$ is the residual mapping in \ref{['eq:direction:Rgam']}. Both AA-I and AA-II use memory parameter $m=10$ for \ref{['fig:bmg:250']} and $m=20$ for \ref{['fig:bmg:500']}. The number of operator evaluations equals the number of $F$ and ${\mathrm{prox}}_{\gamma g}$ evaluations.
• Figure 3: Convergence of algorithms on the Cournot–Nash equilibrium problem where $r^{k} = R_{1/2L_F}(z^k)$ and $R$ is the residual mapping in \ref{['eq:direction:Rgam']}. Both AA-I and AA-II use memory parameter $m=3$. The number of operator evaluations equals the number of $F$ and ${\mathrm{prox}}_{\gamma g}$ evaluations.
• Figure 4: Convergence of algorithms on the sparse logistic regression problem \ref{['eq:logistic_01_label']}, using the datasets a9a from chang2011LIBSVM and spambase from Dua2017. Both AA-I and AA-II use memory parameter $m=10$ for \ref{['fig:logistic:a9a:100', 'fig:logistic:a9a:10', 'fig:logistic:a9a:1']} and $m=6$ for \ref{['fig:logistic:spambase:100', 'fig:logistic:spambase:10', 'fig:logistic:spambase:1']}. The number of operator evaluations equals the number of $F$ and ${\mathrm{prox}}_{\gamma g}$ evaluations.

Theorems & Definitions (53)

• Proposition 2.1
• proof
• Theorem 2.2
• proof
• Corollary 2.3
• proof
• Theorem 2.4
• proof
• Corollary 2.5
• proof