Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Accelerated Gradient and Skew-Symmetric Splitting Methods for a Class of Monotone Operator Equations

Long Chen, Jingrong Wei

TL;DR

This work develops Gradient and Skew-Symmetric Splitting (GSS) methods and their Accelerated variants (AGSS) for solving monotone operator equations A(x)=0 with A(x) = ∇F(x) + N x, where F is strongly convex and N is skew-symmetric. By discretizing a generalized gradient flow and employing a split of the skew-symmetric part, the authors obtain explicit and implicit schemes with linear convergence, and they further enhance convergence through accelerated gradient flow, IMEX schemes, and AOR-based updates. The methods extend to nonlinear saddle-point systems with bilinear coupling, achieving optimal first-order iteration complexity under suitable preconditioners, and they demonstrate robustness across quadratic problems, convection-diffusion models, and empirical risk minimization. The results are supported by Lyapunov-based analyses, comparisons to HSS, and a range of numerical experiments, highlighting practical efficiency and scalability in both linear and nonlinear settings.

Abstract

A class of monotone operator equations, which can be decomposed into sum of the gradient of a strongly convex function and a linear and skew-symmetric operator, is considered in this work. Based on discretization of the generalized gradient flow, gradient and skew-symmetric splitting (GSS) methods are proposed and proved to converge in linear rates. To further accelerate the convergence, an accelerated gradient flow is proposed and accelerated gradient and skew-symmetric splitting (AGSS) methods are developed, which extends the acceleration among the existing works on the convex minimization to a more general class of monotone operator equations. In particular, when applied to smooth saddle point systems with bilinear coupling, a linear convergent method with optimal lower iteration complexity is proposed. The robustness and efficiency of GSS and AGSS methods are verified via extensive numerical experiments.

Accelerated Gradient and Skew-Symmetric Splitting Methods for a Class of Monotone Operator Equations

TL;DR

This work develops Gradient and Skew-Symmetric Splitting (GSS) methods and their Accelerated variants (AGSS) for solving monotone operator equations A(x)=0 with A(x) = ∇F(x) + N x, where F is strongly convex and N is skew-symmetric. By discretizing a generalized gradient flow and employing a split of the skew-symmetric part, the authors obtain explicit and implicit schemes with linear convergence, and they further enhance convergence through accelerated gradient flow, IMEX schemes, and AOR-based updates. The methods extend to nonlinear saddle-point systems with bilinear coupling, achieving optimal first-order iteration complexity under suitable preconditioners, and they demonstrate robustness across quadratic problems, convection-diffusion models, and empirical risk minimization. The results are supported by Lyapunov-based analyses, comparisons to HSS, and a range of numerical experiments, highlighting practical efficiency and scalability in both linear and nonlinear settings.

Abstract

A class of monotone operator equations, which can be decomposed into sum of the gradient of a strongly convex function and a linear and skew-symmetric operator, is considered in this work. Based on discretization of the generalized gradient flow, gradient and skew-symmetric splitting (GSS) methods are proposed and proved to converge in linear rates. To further accelerate the convergence, an accelerated gradient flow is proposed and accelerated gradient and skew-symmetric splitting (AGSS) methods are developed, which extends the acceleration among the existing works on the convex minimization to a more general class of monotone operator equations. In particular, when applied to smooth saddle point systems with bilinear coupling, a linear convergent method with optimal lower iteration complexity is proposed. The robustness and efficiency of GSS and AGSS methods are verified via extensive numerical experiments.
Paper Structure (36 sections, 19 theorems, 196 equations, 1 figure, 9 tables)

This paper contains 36 sections, 19 theorems, 196 equations, 1 figure, 9 tables.

Table of Contents

  1. Introduction
  2. Contribution
  3. Gradient and Skew-symmetric Splitting Methods
  4. Accelerated Gradient and Skew-symmetric Splitting Methods
  5. Nonlinear Saddle Point Systems with bilinear coupling
  6. Related Work
  7. Preliminary
  8. Convex Functions
  9. Inner Products and Preconditioners
  10. Lyapunov Analysis
  11. Gradient and skew-Symmetric Splitting Methods
  12. Stability
  13. Implicit Euler Schemes
  14. Explicit Euler Schemes
  15. Accelerated Overrelaxation Methods for Shifted skew-Symmetric Problems
  16. ...and 21 more sections

Key Result

Lemma 2.1

If function $F: \mathcal{X} \rightarrow \mathbb{R}$ is differentiable, then for any $x, y, z \in \mathcal{X}$, it holds that

Figures (1)

  • Figure 1: Convergence rate of GSS methods and AGSS methods.

Theorems & Definitions (31)

  • Lemma 2.1: Bregman divergence identity chen1993convergence
  • Theorem 3.1
  • proof
  • Lemma 3.2
  • proof
  • Theorem 3.3
  • proof
  • Theorem 3.4
  • Lemma 4.1
  • proof
  • ...and 21 more