Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Linear Convergence of a Unified Primal--Dual Algorithm for Convex--Concave Saddle Point Problems with Quadratic Growth

Cody Melcher, Afrooz Jalilzadeh, Erfan Yazdandoost Hamedani

TL;DR

This paper proposes a generalized accelerated primal-dual (GAPD) algorithm to solve saddle point (SP) problems with non-bilinear objective functions, unifying and extending existing methods and proves that the method achieves a linear convergence rate under relaxed conditions.

Abstract

In this paper, we study saddle point (SP) problems, focusing on convex-concave optimization involving functions that satisfy either two-sided quadratic functional growth (QFG) or two-sided quadratic gradient growth (QGG)--novel conditions tailored specifically for SP problems as extensions of quadratic growth conditions in minimization. These conditions relax the traditional requirement of strong convexity-strong concavity, thereby encompassing a broader class of problems. We propose a generalized accelerated primal-dual (GAPD) algorithm to solve SP problems with non-bilinear objective functions, unifying and extending existing methods. We prove that our method achieves a linear convergence rate under these relaxed conditions. Additionally, we provide examples of structured SP problems that satisfy either two-sided QFG or QGG, demonstrating the practical applicability and relevance of our approach.

Linear Convergence of a Unified Primal--Dual Algorithm for Convex--Concave Saddle Point Problems with Quadratic Growth

TL;DR

This paper proposes a generalized accelerated primal-dual (GAPD) algorithm to solve saddle point (SP) problems with non-bilinear objective functions, unifying and extending existing methods and proves that the method achieves a linear convergence rate under relaxed conditions.

Abstract

In this paper, we study saddle point (SP) problems, focusing on convex-concave optimization involving functions that satisfy either two-sided quadratic functional growth (QFG) or two-sided quadratic gradient growth (QGG)--novel conditions tailored specifically for SP problems as extensions of quadratic growth conditions in minimization. These conditions relax the traditional requirement of strong convexity-strong concavity, thereby encompassing a broader class of problems. We propose a generalized accelerated primal-dual (GAPD) algorithm to solve SP problems with non-bilinear objective functions, unifying and extending existing methods. We prove that our method achieves a linear convergence rate under these relaxed conditions. Additionally, we provide examples of structured SP problems that satisfy either two-sided QFG or QGG, demonstrating the practical applicability and relevance of our approach.

Paper Structure

This paper contains 12 sections, 8 theorems, 77 equations, 1 figure, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Contribution
  4. Assumptions and Definitions
  5. Notation
  6. Proposed Method
  7. Convergence Analysis
  8. Class of problems satisfying two-sided QFG/QGG
  9. Numerical Experiment
  10. Conclusion and Future Directions
  11. Proof of Corollary \ref{['cor:stepresult']}
  12. Proof of Descent Inequality for Dual Iterates

Key Result

Theorem 2.1

Suppose Assumption assumption:convex-concave-smoothness holds and the Bregman distance generating functions $\psi_\mathcal{X}$ and $\psi_\mathcal{Y}$ has Lipschitz continuous gradient with constants $L_{\psi_\mathcal{X}}$ and $L_{\psi_\mathcal{Y}}$, respectively. If $f$ satisfies the two-sided QGG i

Figures (1)

  • Figure 1: Convergence of GDA and GAPD variants on problem \ref{['eq:quadratic-sp']}, measured by the normalized gradient residual. From left to right, the problem dimensions are: $(n,m,p,q)= (75,60,60,50),(150,120,120,100),(300,240,240,200)$.

Theorems & Definitions (30)

  • Definition 2.1: Bregman Distance
  • Remark 2.1
  • Definition 2.2: Bregman Projection
  • Definition 2.3: QGG
  • Definition 2.4: QFG
  • Definition 2.5: Saddle Point Solution
  • Definition 2.6: Two-Sided QGG
  • Definition 2.7: Two-Sided QFG
  • Remark 2.2
  • Theorem 2.1
  • ...and 20 more