Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convergence of Descent Optimization Algorithms under Polyak-Łojasiewicz-Kurdyka Conditions

G. C. Bento, B. S. Mordukhovich, T. S. Mota, Yu. Nesterov

TL;DR

The paper addresses convergence of descent algorithms for nonsmooth, nonconvex optimization under PLK-type conditions, including lower-exponent variants. It develops a unified global convergence and rate analysis for two generic descent classes under two error frameworks, and applies the results to inexact reduced gradient methods and the Boosted DC Algorithm, deriving finite termination and various rate regimes depending on the PLK exponent. A key contribution is the demonstration of finite termination for lower-exponent PLK and detailed rate characterizations across exponent ranges, along with an inconsistency result for difference programs with Lipschitz gradients. The findings provide a rigorous theoretical foundation for a broad class of descent methods in nonsmooth optimization, clarifying limitations in certain DC-like settings and guiding algorithm design and analysis.

Abstract

This paper develops a comprehensive convergence analysis for generic classes of descent algorithms in nonsmooth and nonconvex optimization under several conditions of the Polyak-Łojasiewicz-Kurdyka (PLK) type. Along other results, we prove the finite termination of generic algorithms under the PLK conditions with lower exponents. Specifications are given to establish new convergence rates for inexact reduced gradient methods and some versions of the boosted algorithm in DC programming. It is revealed, e.g., that the lower exponent PLK conditions for a broad class of difference programs are incompatible with the gradient Lipschitz continuity for the plus function around a local minimizer. On the other hand, we show that the above inconsistency observation may fail if the Lipschitz continuity is replaced by merely the gradient continuity.

Convergence of Descent Optimization Algorithms under Polyak-Łojasiewicz-Kurdyka Conditions

TL;DR

The paper addresses convergence of descent algorithms for nonsmooth, nonconvex optimization under PLK-type conditions, including lower-exponent variants. It develops a unified global convergence and rate analysis for two generic descent classes under two error frameworks, and applies the results to inexact reduced gradient methods and the Boosted DC Algorithm, deriving finite termination and various rate regimes depending on the PLK exponent. A key contribution is the demonstration of finite termination for lower-exponent PLK and detailed rate characterizations across exponent ranges, along with an inconsistency result for difference programs with Lipschitz gradients. The findings provide a rigorous theoretical foundation for a broad class of descent methods in nonsmooth optimization, clarifying limitations in certain DC-like settings and guiding algorithm design and analysis.

Abstract

This paper develops a comprehensive convergence analysis for generic classes of descent algorithms in nonsmooth and nonconvex optimization under several conditions of the Polyak-Łojasiewicz-Kurdyka (PLK) type. Along other results, we prove the finite termination of generic algorithms under the PLK conditions with lower exponents. Specifications are given to establish new convergence rates for inexact reduced gradient methods and some versions of the boosted algorithm in DC programming. It is revealed, e.g., that the lower exponent PLK conditions for a broad class of difference programs are incompatible with the gradient Lipschitz continuity for the plus function around a local minimizer. On the other hand, we show that the above inconsistency observation may fail if the Lipschitz continuity is replaced by merely the gradient continuity.
Paper Structure (9 sections, 10 theorems, 118 equations, 1 algorithm)

This paper contains 9 sections, 10 theorems, 118 equations, 1 algorithm.

Table of Contents

  1. Introduction, Basic Definitions, and Discussions
  2. Global Convergence of Generic Algorithms
  3. Convergence Rates under Relative Error Condition
  4. Convergence Rates under Modified Error Property
  5. Specifications of Generic Descent Algorithms
  6. Inexact Reduced Graduate Methods
  7. Boosted DC Algorithm
  8. Lower Exponent PLK for Difference Programs
  9. Conclusions

Key Result

Theorem 1

Let $\phi :\mathbb{R}^n \longrightarrow\overline{\mathbb{R}}$ be a proper l.s.c. function bounded from below, and let the sequence $\{x^k\}$ be constructed by the generic algorithms satisfying either $(\mathcal{H}1)$ and $(\mathcal{H}2)$, or $(\mathcal{H}1)$ and $(\mathcal{H}3)$ properties. If the b and that $\{x^k\}$ converges to $\Bar{x}$ as $k\to\infty$. Moreover, $\Bar{x}$ is an $M$-stationary

Theorems & Definitions (27)

  • Definition 1: PLK conditions
  • Theorem 1: global convergence of generic algorithms under the basic PLK
  • proof
  • Remark 1: convergence under symmetric and strong PLK conditions
  • Lemma 1: estimates for monotone sequences
  • proof
  • Lemma 2: convergence rates for monotone sequences
  • proof
  • Theorem 2: value convergence rates under exponent PLK and relative error conditions
  • proof
  • ...and 17 more