Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Inertial Bregman Proximal Gradient under Partial Smoothness

Jean-Jacques Godeme

TL;DR

The paper develops an Inertial Bregman Proximal Gradient (IBPG) method for minimizing $\Phi(x)=F(x)+G(x)$ in finite dimensions, where $F$ may be nonconvex and $G$ is convex. By leveraging a Bregman entropy $\psi$ to realize relative smoothness and the triangle scaling property (TSP), IBPG achieves acceleration in a broad non-Euclidean geometry, with global convergence guaranteed under the Kurdyka-Łojasiewicz (KL) framework when the entropy is strongly convex. Locally, if the nonsmooth part is partly smooth relative to a smooth manifold, the method enjoys finite activity identification and local linear convergence via a spectral analysis of a linearized map; in the smooth case, the approach also exhibits trap-avoidance properties around strict saddles. The authors illustrate the theory with phase retrieval experiments under various regularizers, demonstrating finite active-set identification and rates consistent with the developed theory, highlighting the practical impact for structured nonsmooth optimization in imaging and inverse problems.

Abstract

This work considers an Inertial version of Bregman Proximal Gradient algorithm (IBPG) for minimizing the sum of two single-valued functions in finite dimension. We suppose that one of the functions is proper, closed, and convex but non-necessarily smooth whilst the second is a smooth enough function but not necessarily convex. For the latter, we ask the smooth adaptable property (smad) with respect to some kernel or entropy which allows to remove the very popular global Lipschitz continuity requirement on the gradient of the smooth part. We consider the IBPG under the framework of the triangle scaling property (TSP) which is a geometrical property for which one can provably ensure acceleration for a certain subset of kernel/entropy functions in the convex setting. Based on this property, we provide global convergence guarantees when the entropy is strongly convex under the framework of the Kurdyka-Łojasiewicz (KL) property. Turning to the local convergence properties, we show that when the nonsmooth part is partly smooth relative to a smooth submanifold, IBPG has a finite activity identification property before entering a local linear convergence regime for which we establish a sharp estimate of the convergence rate. We report numerical simulations to illustrate our theoretical results on low complexity regularized phase retrieval.

Inertial Bregman Proximal Gradient under Partial Smoothness

TL;DR

The paper develops an Inertial Bregman Proximal Gradient (IBPG) method for minimizing in finite dimensions, where may be nonconvex and is convex. By leveraging a Bregman entropy to realize relative smoothness and the triangle scaling property (TSP), IBPG achieves acceleration in a broad non-Euclidean geometry, with global convergence guaranteed under the Kurdyka-Łojasiewicz (KL) framework when the entropy is strongly convex. Locally, if the nonsmooth part is partly smooth relative to a smooth manifold, the method enjoys finite activity identification and local linear convergence via a spectral analysis of a linearized map; in the smooth case, the approach also exhibits trap-avoidance properties around strict saddles. The authors illustrate the theory with phase retrieval experiments under various regularizers, demonstrating finite active-set identification and rates consistent with the developed theory, highlighting the practical impact for structured nonsmooth optimization in imaging and inverse problems.

Abstract

This work considers an Inertial version of Bregman Proximal Gradient algorithm (IBPG) for minimizing the sum of two single-valued functions in finite dimension. We suppose that one of the functions is proper, closed, and convex but non-necessarily smooth whilst the second is a smooth enough function but not necessarily convex. For the latter, we ask the smooth adaptable property (smad) with respect to some kernel or entropy which allows to remove the very popular global Lipschitz continuity requirement on the gradient of the smooth part. We consider the IBPG under the framework of the triangle scaling property (TSP) which is a geometrical property for which one can provably ensure acceleration for a certain subset of kernel/entropy functions in the convex setting. Based on this property, we provide global convergence guarantees when the entropy is strongly convex under the framework of the Kurdyka-Łojasiewicz (KL) property. Turning to the local convergence properties, we show that when the nonsmooth part is partly smooth relative to a smooth submanifold, IBPG has a finite activity identification property before entering a local linear convergence regime for which we establish a sharp estimate of the convergence rate. We report numerical simulations to illustrate our theoretical results on low complexity regularized phase retrieval.

Paper Structure

This paper contains 46 sections, 22 theorems, 64 equations, 4 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Problem statement
  3. Prior work
  4. Inertial Bregman proximal gradient algorithms
  5. Activity identification
  6. Strict saddle avoidance
  7. Contributions
  8. Global convergence of the scheme
  9. Finite activity identification
  10. Local linear convergence analysis
  11. Escape property in the smooth case
  12. Paper organization
  13. Preliminaries
  14. Notations
  15. General Definitions
  16. ...and 31 more sections

Key Result

Theorem 3.2

Consider problem (eq: generalpro) under the Premises assump_A and assump_B. Suppose that the sequence of IBPG parameters $\left(a_{k}\right)_{k\in\mathbb{N}}$ are chosen as in Algorithm alg:BPGBT. Then,

Figures (4)

  • Figure 1: Phase retrieval by solving (\ref{['eq:phase_retrieval']}) with the $\ell_1-$norm regularizer.
  • Figure 2: Phase retrieval by solving (\ref{['eq:phase_retrieval']}) with the $\ell_{1,2}-$norm regularizer.
  • Figure 3: Phase retrieval by solving (\ref{['eq:phase_retrieval']}) with the TV semi-norm.
  • Figure 4: Phase retrieval with the synthesis prior formulation.

Theorems & Definitions (54)

  • Definition 1.1
  • Definition 2.1
  • Definition 2.2
  • Remark 2.3
  • Definition 2.4
  • Remark 2.5
  • Definition 2.6
  • Remark 2.7
  • Remark 3.1
  • Theorem 3.2
  • ...and 44 more