Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Proximal gradient-type method with generalized distance and convergence analysis without global descent lemma

Shotaro Yagishita, Masaru Ito

TL;DR

This work addresses nonconvex composite optimization $F(x)=f(x)+g(x)$ by introducing a generalized variable distance proximal gradient method (GVDPGM) that permits iteration-dependent prox-distance terms $D_k$ and uses backtracking to satisfy an average-type Armijo condition, removing the need for a global descent lemma. Through KL-based analysis, the authors establish whole-sequence convergence to a critical point and characterize convergence rates in terms of the desingularization function $\chi(t)=ct^\theta$, yielding finite steps for $\theta=1$, Q-/R-superlinear behavior for $1/2<\theta<1$, and linear/sublinear rates for other regimes. The paper also presents two applications: a trimmed logistic regression model using an exponential prox-grad distance and interior gradient methods for conic optimization with non-Bregman distances, including nonnegative orthant, PSD/second-order cones, and unit simplex, with convergence to d-stationary points even on boundaries. These results broaden the design space of proximal-gradient-type algorithms by focusing on compatibility with $g$ rather than the smooth term $f$, enabling efficient subproblem solutions and applicable convergence guarantees in nonconvex settings. The theoretical framework and applications suggest practical pathways for robust, scalable optimization in machine learning and conic optimization contexts.

Abstract

We consider solving nonconvex composite optimization problems in which the sum of a smooth function and a nonsmooth function is minimized. Many of convergence analyses of proximal gradient-type methods rely on global descent property between the smooth term and its proximal term. On the other hand, the ability to efficiently solve the subproblem depends on the compatibility between the nonsmooth term and the proximal term. Selecting an appropriate proximal term by considering both factors simultaneously is generally difficult. We overcome this issue by providing convergence analyses for proximal gradient-type methods with general proximal terms, without requiring global descent property of the smooth term. As a byproduct, new convergence results of the interior gradient methods for conic optimization are also provided.

Proximal gradient-type method with generalized distance and convergence analysis without global descent lemma

TL;DR

This work addresses nonconvex composite optimization by introducing a generalized variable distance proximal gradient method (GVDPGM) that permits iteration-dependent prox-distance terms and uses backtracking to satisfy an average-type Armijo condition, removing the need for a global descent lemma. Through KL-based analysis, the authors establish whole-sequence convergence to a critical point and characterize convergence rates in terms of the desingularization function , yielding finite steps for , Q-/R-superlinear behavior for , and linear/sublinear rates for other regimes. The paper also presents two applications: a trimmed logistic regression model using an exponential prox-grad distance and interior gradient methods for conic optimization with non-Bregman distances, including nonnegative orthant, PSD/second-order cones, and unit simplex, with convergence to d-stationary points even on boundaries. These results broaden the design space of proximal-gradient-type algorithms by focusing on compatibility with rather than the smooth term , enabling efficient subproblem solutions and applicable convergence guarantees in nonconvex settings. The theoretical framework and applications suggest practical pathways for robust, scalable optimization in machine learning and conic optimization contexts.

Abstract

We consider solving nonconvex composite optimization problems in which the sum of a smooth function and a nonsmooth function is minimized. Many of convergence analyses of proximal gradient-type methods rely on global descent property between the smooth term and its proximal term. On the other hand, the ability to efficiently solve the subproblem depends on the compatibility between the nonsmooth term and the proximal term. Selecting an appropriate proximal term by considering both factors simultaneously is generally difficult. We overcome this issue by providing convergence analyses for proximal gradient-type methods with general proximal terms, without requiring global descent property of the smooth term. As a byproduct, new convergence results of the interior gradient methods for conic optimization are also provided.
Paper Structure (13 sections, 16 theorems, 130 equations, 1 algorithm)

This paper contains 13 sections, 16 theorems, 130 equations, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related works
  3. Notation and Preliminaries
  4. Generalized variable distance proximal gradient method
  5. Global convergence and rate of convergence under KL assumption
  6. Applications
  7. Exponential prox-grad distance for trimmed logistic regression
  8. Interior gradient method for conic optimization
  9. Nonnegative orthant
  10. Positive semidefinite cone and second-order cone
  11. Unit simplex
  12. Conclusion
  13. Proofs

Key Result

Proposition 1.1

Let $\phi:\mathbb{E}\to(-\infty,\infty]$ be lower semicontinuous and prox-bounded with respect to a prox-grad distance $D$ with threshold $\gamma_{\phi,D}$. Suppose that $\mathop{\rm dom}\phi$ is included in $\mathop{\rm cl}\mathcal{C}$ and $\mathcal{C}\cap\mathop{\rm dom}\phi$ is nonempty. Then, fo is nonempty and compact.

Theorems & Definitions (39)

  • Definition 1.1: attouch2010proximalattouch2013convergencebolte2014proximal
  • Definition 1.2
  • Definition 1.3
  • Proposition 1.1
  • proof
  • Lemma 2.1
  • proof
  • Proposition 2.1
  • proof
  • Lemma 2.2
  • ...and 29 more