Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Accelerated Extragradient-Type Methods -- Part 2: Generalization and Sublinear Convergence Rates under Co-Hypomonotonicity

Quoc Tran-Dinh, Nghia Nguyen-Trung

Abstract

Following the first part of our project, this paper comprehensively studies two types of extragradient-based methods: anchored extragradient and Nesterov's accelerated extragradient for solving [non]linear inclusions (and, in particular, equations), primarily under the Lipschitz continuity and the co-hypomonotonicity assumptions. We unify and generalize a class of anchored extragradient methods for monotone inclusions to a wider range of schemes encompassing existing algorithms as special cases. We establish $\mathcal{O}(1/k)$ last-iterate convergence rates on the residual norm of the underlying mapping for this general framework and then specialize it to obtain convergence guarantees for specific instances, where $k$ denotes the iteration counter. We extend our approach to a class of anchored Tseng's forward-backward-forward splitting methods to obtain a broader class of algorithms for solving co-hypomonotone inclusions. Again, we analyze $\mathcal{O}(1/k)$ last-iterate convergence rates for this general scheme and specialize it to obtain convergence results for existing and new variants. We generalize and unify Nesterov's accelerated extra-gradient method to a new class of algorithms that covers existing schemes as special instances while generating new variants. For these schemes, we can prove $\mathcal{O}(1/k)$ last-iterate convergence rates for the residual norm under co-hypomonotonicity, covering a class of nonmonotone problems. We propose another novel class of Nesterov's accelerated extragradient methods to solve inclusions. Interestingly, these algorithms achieve both $\mathcal{O}(1/k)$ and $o(1/k)$ last-iterate convergence rates, and also the convergence of iterate sequences under co-hypomonotonicity and Lipschitz continuity. Finally, we provide a set of numerical experiments encompassing different scenarios to validate our algorithms and theoretical guarantees.

Accelerated Extragradient-Type Methods -- Part 2: Generalization and Sublinear Convergence Rates under Co-Hypomonotonicity

Abstract

Following the first part of our project, this paper comprehensively studies two types of extragradient-based methods: anchored extragradient and Nesterov's accelerated extragradient for solving [non]linear inclusions (and, in particular, equations), primarily under the Lipschitz continuity and the co-hypomonotonicity assumptions. We unify and generalize a class of anchored extragradient methods for monotone inclusions to a wider range of schemes encompassing existing algorithms as special cases. We establish last-iterate convergence rates on the residual norm of the underlying mapping for this general framework and then specialize it to obtain convergence guarantees for specific instances, where denotes the iteration counter. We extend our approach to a class of anchored Tseng's forward-backward-forward splitting methods to obtain a broader class of algorithms for solving co-hypomonotone inclusions. Again, we analyze last-iterate convergence rates for this general scheme and specialize it to obtain convergence results for existing and new variants. We generalize and unify Nesterov's accelerated extra-gradient method to a new class of algorithms that covers existing schemes as special instances while generating new variants. For these schemes, we can prove last-iterate convergence rates for the residual norm under co-hypomonotonicity, covering a class of nonmonotone problems. We propose another novel class of Nesterov's accelerated extragradient methods to solve inclusions. Interestingly, these algorithms achieve both and last-iterate convergence rates, and also the convergence of iterate sequences under co-hypomonotonicity and Lipschitz continuity. Finally, we provide a set of numerical experiments encompassing different scenarios to validate our algorithms and theoretical guarantees.
Paper Structure (71 sections, 38 theorems, 391 equations, 9 figures, 1 table)

This paper contains 71 sections, 38 theorems, 391 equations, 9 figures, 1 table.

Table of Contents

  1. Introduction
  2. Problem statements, equivalent forms, and special cases
  3. Our goals and contributions
  4. Related work
  5. Paper organization
  6. Background and Preliminary Results
  7. Basic Concepts, Monotonicity, and Lipschitz Continuity
  8. Exact Solutions and Approximate Solutions
  9. A Class of Extra-Anchored Gradient Methods for Monotone \ref{['eq:NI']}
  10. A Class of Extra-Anchored Gradient Methods for \ref{['eq:NI']}
  11. Key Estimates for Convergence Analysis
  12. Convergence Analysis of \ref{['eq:EAG4NI']} and Its Special Instances
  13. A Class of Anchored FBFS Methods for Inclusion \ref{['eq:NI']}
  14. A Class of Anchored FBFS Methods
  15. Key Estimates for Convergence Analysis
  16. ...and 56 more sections

Key Result

lemma thmcounterlemma

For eq:NI, assume that $\mathrm{zer}(\Phi) \neq\emptyset$, $F$ is $L$-Lipschitz continuous and monotone, and $T$ is maximally $3$-cyclically monotone. Let $\{(x^k, y^k)\}$ be generated by eq:EAG4NI satisfying eq:EAG4NI_uk_cond and $\mathcal{V}_k$ be defined by eq:EAG4NI_potential_func. Suppose that Then, for any $\omega > 0$, $r > 0$, and $c > 0$, let $M_c := (1+c)(1+\omega)L^2$, we have

Figures (9)

  • Figure 1: The search directions in existing methods (EG- and Past-EG-type) and our schemes (GEG-type): with $u^k := \alpha_1 Fx^k + \alpha_2 Fy^{k-1} + \alpha_3 v^{k-1}$ for $\alpha_1 = 1$, $\alpha_2=0.5$, and $\alpha_3 = -0.5$.
  • Figure 2: The behaviors of the first set of algorithms for solving \ref{['eq:NI']} in Experiments 1 and 2 when we choose $u^k := Fx^k$. The plot reveals the mean of 10 problem instances.
  • Figure 3: The behaviors of the second set of algorithms for solving \ref{['eq:NI']} in Experiments 1 and 2 when we choose $u^k := Fy^{k-1}$. The plot reveals the mean of 10 problem instances.
  • Figure 4: The behaviors of \ref{['eq:DFEG4NI']} for solving \ref{['eq:NI']} in Experiments 1 and 2. The plot reveals the mean of 10 problem instances. The legend presents the values of $\mu$ used in the corresponding instant of \ref{['eq:DFEG4NI']}.
  • Figure 5: The behaviors of \ref{['eq:NGEAG4NI']} for solving \ref{['eq:NI']} in Experiments 1 and 2. The plot reveals the mean of 10 problem instances. The legend presents the values of $r$ and $\mu$ used in the corresponding instant of \ref{['eq:NGEAG4NI']}. For example, $(3, 1)$ means the solid orange line with filled circle marker corresponds to the instance of \ref{['eq:NGEAG4NI']} using $r = 3$ and $\mu = 1$.
  • ...and 4 more figures

Theorems & Definitions (54)

  • remark thmcounterremark
  • lemma thmcounterlemma
  • theorem 1
  • corollary thmcountercorollary
  • proof
  • remark thmcounterremark
  • lemma thmcounterlemma
  • lemma thmcounterlemma
  • lemma thmcounterlemma
  • theorem 2
  • ...and 44 more