Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A nonmonotone extrapolated proximal gradient-subgradient algorithm beyond global Lipschitz gradient continuity

Lei Yang, Jingjing Hu, Tianxiang Liu

TL;DR

This work tackles composite optimization problems where the smooth part lacks a globally Lipschitz gradient. It introduces nexPGA, an extrapolated proximal gradient–subgradient method with a ZH-type nonmonotone line search that is problem-parameter-free. The authors establish global convergence and rates via the KL property without requiring bounded iterates, extending analysis to non-Lipschitz settings and extrapolation. Numerical experiments on large-scale l1-2 regularized problems show that nexPGA achieves accelerated convergence compared with several state-of-the-art proximal methods, highlighting practical benefits of combining extrapolation with nonmonotone descent. The results broaden the theoretical and practical applicability of PG-type algorithms in non-Lipschitz regimes.

Abstract

With the advancement of modern applications, an increasing number of composite optimization problems arise whose smooth component does not possess a globally Lipschitz continuous gradient. This setting prevents the direct use of the proximal gradient (PG) method and its variants, and has motivated a growing body of research on new PG-type methods and their convergence theory, in particular, global convergence analysis without imposing any explicit or implicit boundedness assumptions on the iterates. Until recently, the first complete analysis of this kind has been established for the PG method and its specific nonmonotone variants, which has since stimulated further exploration along this research direction. In this paper, we consider a general composite optimization model beyond the global Lipschitz gradient continuity setting. We propose a novel problem-parameter-free algorithm that incorporates a carefully designed nonmonotone line search to handle the non-global Lipschitz gradient continuity, together with an extrapolation step to achieve potential acceleration. Despite the added technical challenges introduced by combining extrapolation with nonmonotone line search, we establish a refined convergence analysis for the proposed algorithm under the Kurdyka-Ł ojasiewicz property, without requiring any boundedness assumptions on the iterates. This work thus further advances the theoretical understanding of PG-type methods in the non-global Lipschitz gradient continuity setting. Finally, we conduct numerical experiments to illustrate the effectiveness of our algorithm and highlight the advantages of integrating extrapolation with a nonmonotone line search.

A nonmonotone extrapolated proximal gradient-subgradient algorithm beyond global Lipschitz gradient continuity

TL;DR

This work tackles composite optimization problems where the smooth part lacks a globally Lipschitz gradient. It introduces nexPGA, an extrapolated proximal gradient–subgradient method with a ZH-type nonmonotone line search that is problem-parameter-free. The authors establish global convergence and rates via the KL property without requiring bounded iterates, extending analysis to non-Lipschitz settings and extrapolation. Numerical experiments on large-scale l1-2 regularized problems show that nexPGA achieves accelerated convergence compared with several state-of-the-art proximal methods, highlighting practical benefits of combining extrapolation with nonmonotone descent. The results broaden the theoretical and practical applicability of PG-type algorithms in non-Lipschitz regimes.

Abstract

With the advancement of modern applications, an increasing number of composite optimization problems arise whose smooth component does not possess a globally Lipschitz continuous gradient. This setting prevents the direct use of the proximal gradient (PG) method and its variants, and has motivated a growing body of research on new PG-type methods and their convergence theory, in particular, global convergence analysis without imposing any explicit or implicit boundedness assumptions on the iterates. Until recently, the first complete analysis of this kind has been established for the PG method and its specific nonmonotone variants, which has since stimulated further exploration along this research direction. In this paper, we consider a general composite optimization model beyond the global Lipschitz gradient continuity setting. We propose a novel problem-parameter-free algorithm that incorporates a carefully designed nonmonotone line search to handle the non-global Lipschitz gradient continuity, together with an extrapolation step to achieve potential acceleration. Despite the added technical challenges introduced by combining extrapolation with nonmonotone line search, we establish a refined convergence analysis for the proposed algorithm under the Kurdyka-Ł ojasiewicz property, without requiring any boundedness assumptions on the iterates. This work thus further advances the theoretical understanding of PG-type methods in the non-global Lipschitz gradient continuity setting. Finally, we conduct numerical experiments to illustrate the effectiveness of our algorithm and highlight the advantages of integrating extrapolation with a nonmonotone line search.

Paper Structure

This paper contains 9 sections, 10 theorems, 124 equations, 1 figure, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Notation and preliminaries
  3. A nonmonotone extrapolated proximal gradient-subgradient algorithm
  4. The KL-based convergence analysis
  5. Numerical experiments
  6. Conclusion
  7. Missing Proofs in Section \ref{['sec-kl-analysis']}
  8. Proof of Theorem \ref{['theorem-fun-rate']}
  9. Proof of Theorem \ref{['theorem-seq-rate']}

Key Result

Proposition 2.1

Suppose that $h: \mathbb{R}^n \rightarrow \mathbb{R} \cup \{+\infty\}$ is a proper closed function and $\Gamma$ is a compact set. If $h \equiv \zeta$ on $\Gamma$ for some constant $\zeta$ and satisfies the KL property at each point of $\Gamma$, then there exist $\varepsilon>0$, $\nu>0$ and $\varphi for all $\bm{x} \in \{\bm{x}\in\mathbb{R}^{n}: \mathrm{dist}(\bm{x},\,\Gamma)<\varepsilon\} \cap \{

Figures (1)

  • Figure 1: The average $E(t)$ of 10 independent trials of different methods for solving \ref{['probleml12']}.

Theorems & Definitions (24)

  • Definition 2.1: KL property/function
  • Definition 2.2: KL exponent
  • Proposition 2.1: Uniformized KL property
  • Definition 3.1: Stationary point
  • Lemma 3.1: Asymptotic sufficient descent property
  • proof
  • Lemma 3.2: Well-definedness of the line search criterion \ref{['nexPGA-lscond']}
  • proof
  • Proposition 3.1
  • proof
  • ...and 14 more