Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Approximate Bregman proximal gradient algorithm with variable metric Armijo--Wolfe line search

Kiwamu Fujiki, Shota Takahashi, Akiko Takeda

TL;DR

This work addresses composite nonconvex optimization of the form $\min_x f(x)+g(x)$ with $f$ smooth and potentially non-Lipschitz, proposing the ABPG-VMAW method that combines an approximate Bregman proximal step with a variable-metric Armijo--Wolfe line search. The authors establish global subsequential convergence and, under the KL property, global convergence to a stationary point, even when $g$ is nonzero. They demonstrate practical effectiveness on $\ell_p$ regularized least squares and nonnegative linear inverse problems, where ABPG-VMAW outperforms ABPG and standard proximal gradient methods. The results advance ABPG-type algorithms by enabling larger, more robust steps while preserving convergence guarantees, with implications for efficiency in signal processing and machine learning applications that involve nonconvex penalties and constraints.

Abstract

We propose a variant of the approximate Bregman proximal gradient (ABPG) algorithm for minimizing the sum of a smooth nonconvex function and a nonsmooth convex function. Although ABPG is known to converge globally to a stationary point even when the smooth part of the objective function lacks globally Lipschitz continuous gradients, and its iterates can often be expressed in closed form, ABPG relies on an Armijo line search to guarantee global convergence. Such reliance can slow down performance in practice. To overcome this limitation, we propose the ABPG with a variable metric Armijo--Wolfe line search. Under the variable metric Armijo--Wolfe condition, we establish the global subsequential convergence of our algorithm. Moreover, assuming the Kurdyka--Łojasiewicz property, we also establish that our algorithm globally converges to a stationary point. Numerical experiments on $\ell_p$ regularized least squares problems and nonnegative linear inverse problems demonstrate that our algorithm outperforms existing algorithms.

Approximate Bregman proximal gradient algorithm with variable metric Armijo--Wolfe line search

TL;DR

This work addresses composite nonconvex optimization of the form with smooth and potentially non-Lipschitz, proposing the ABPG-VMAW method that combines an approximate Bregman proximal step with a variable-metric Armijo--Wolfe line search. The authors establish global subsequential convergence and, under the KL property, global convergence to a stationary point, even when is nonzero. They demonstrate practical effectiveness on regularized least squares and nonnegative linear inverse problems, where ABPG-VMAW outperforms ABPG and standard proximal gradient methods. The results advance ABPG-type algorithms by enabling larger, more robust steps while preserving convergence guarantees, with implications for efficiency in signal processing and machine learning applications that involve nonconvex penalties and constraints.

Abstract

We propose a variant of the approximate Bregman proximal gradient (ABPG) algorithm for minimizing the sum of a smooth nonconvex function and a nonsmooth convex function. Although ABPG is known to converge globally to a stationary point even when the smooth part of the objective function lacks globally Lipschitz continuous gradients, and its iterates can often be expressed in closed form, ABPG relies on an Armijo line search to guarantee global convergence. Such reliance can slow down performance in practice. To overcome this limitation, we propose the ABPG with a variable metric Armijo--Wolfe line search. Under the variable metric Armijo--Wolfe condition, we establish the global subsequential convergence of our algorithm. Moreover, assuming the Kurdyka--Łojasiewicz property, we also establish that our algorithm globally converges to a stationary point. Numerical experiments on regularized least squares problems and nonnegative linear inverse problems demonstrate that our algorithm outperforms existing algorithms.

Paper Structure

This paper contains 19 sections, 15 theorems, 78 equations, 8 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Notation
  3. Preliminaries
  4. Subdifferentials
  5. Bregman Distances
  6. Kurdyka--Łojasiewicz Property
  7. Proposed Algorithm: Approximate Bregman Proximal Gradient Algorithm with Variable Metric Armijo--Wolfe Line Search
  8. Approximate Bregman Proximal Gradient Algorithm
  9. Variable Metric Armijo--Wolfe Line Search
  10. Convergence Analysis
  11. Properties of Proposed Algorithm
  12. Global Subsequential Convergence
  13. Global Convergence
  14. Numerical Experiments
  15. lp Regularized Least Squares Problem
  16. ...and 4 more sections

Key Result

Lemma 2.6

Assume that $f:\mathbb{R}^n\rightarrow (-\infty, +\infty]$ is a proper and lower semicontinuous function. If $f$ takes a constant value on some compact set $\Gamma$, and satisfies the KL property on $\Gamma$, then there exist $v,\epsilon \in (0,+\infty ]$, and $\psi\in\Xi_{v}$ such that, for any $\b

Figures (8)

  • Figure 1: Objective function values ($p=1.2$)
  • Figure 2: Accuracy ($p=1.2$)
  • Figure 4: Objective function values ($p=1.2$)
  • Figure 5: Accuracy ($p=1.2$)
  • Figure 7: Objective function values
  • ...and 3 more figures

Theorems & Definitions (34)

  • Definition 2.1: Regular and Limiting Subdifferentials rockafellar2009variational
  • Definition 2.2: Kernal Generating Distances Bolte-2018
  • Definition 2.3: Bregman Distances Bregman1967TheRM
  • Example 2.4
  • Definition 2.5: Kurdyka--Łojasiewicz Property 90074980-9c0c-33c4-b119-70171dab0b45
  • Definition 2.5: Kurdyka--Łojasiewicz Property 90074980-9c0c-33c4-b119-70171dab0b45
  • Lemma 2.6: Uniformized KL property bolte-2014
  • Lemma 3.3: Well-posedness of $\tilde{\mathcal{T}}_{\lambda}$ Takahashi
  • Remark 3.4
  • Proposition 4.2: Search direction property
  • ...and 24 more