Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Inexact Bregman Proximal Difference-of-Convex Algorithm with Two Types of Relative Stopping Criteria

Lei Yang, Jingjing Hu, Kim-Chuan Toh

TL;DR

The work introduces iBPDCA, an inexact Bregman proximal DC algorithm for solving DC problems with restricted $L$-smooth adaptability, featuring two relative stopping criteria (SC1/SC2) to inexactly solve subproblems while preserving convergence under the KL property. Theoretical results establish global subsequential and sequential convergence, including global convergence to $\ell$-stationary points under KL and regularity conditions, with an auxiliary function $H_{\tau}$ enabling SC2 analysis. Numerically, iBPDCA outperforms several state-of-the-art methods on $\ell_{1-2}$ regularized least squares and constrained sparse optimization, using dual semismooth Newton methods to efficiently solve subproblems; SC2 often reduces verification costs for large-scale problems. The framework is flexible across DC decompositions and offers practical, scalable convergence in nonconvex settings, with potential extensions to inexact BPDCAe in future work.

Abstract

In this paper, we consider a class of difference-of-convex (DC) optimization problems, which require only a weaker restricted $L$-smooth adaptable property on the smooth part of the objective function, instead of the standard global Lipschitz gradient continuity assumption. Such problems are prevalent in many contemporary applications such as compressed sensing, statistical regression, and machine learning, and can be solved by a general Bregman proximal DC algorithm (BPDCA). However, the existing BPDCA is developed based on the stringent requirement that the involved subproblems must be solved exactly, which is often impractical and limits the applicability of the BPDCA. To facilitate the practical implementations and wider applications of the BPDCA, we develop an inexact Bregman proximal difference-of-convex algorithm (iBPDCA) by incorporating two types of relative-type stopping criteria for solving the subproblems. The proposed inexact framework has considerable flexibility to encompass many existing exact and inexact methods, and can accommodate different types of errors that may occur when solving the subproblem. This enables the potential application of our inexact framework across different DC decompositions to facilitate the design of a more efficient DCA scheme in practice. The global subsequential convergence and the global sequential convergence of our iBPDCA are established under suitable conditions including the Kurdyka-Łojasiewicz property. Some numerical experiments are conducted to show the superior performance of our iBPDCA in comparison to existing algorithms. These results also empirically validate the necessity and significance of developing different types of stopping criteria to facilitate the efficient computation of the subproblem in each iteration of our iBPDCA.

An Inexact Bregman Proximal Difference-of-Convex Algorithm with Two Types of Relative Stopping Criteria

TL;DR

The work introduces iBPDCA, an inexact Bregman proximal DC algorithm for solving DC problems with restricted -smooth adaptability, featuring two relative stopping criteria (SC1/SC2) to inexactly solve subproblems while preserving convergence under the KL property. Theoretical results establish global subsequential and sequential convergence, including global convergence to -stationary points under KL and regularity conditions, with an auxiliary function enabling SC2 analysis. Numerically, iBPDCA outperforms several state-of-the-art methods on regularized least squares and constrained sparse optimization, using dual semismooth Newton methods to efficiently solve subproblems; SC2 often reduces verification costs for large-scale problems. The framework is flexible across DC decompositions and offers practical, scalable convergence in nonconvex settings, with potential extensions to inexact BPDCAe in future work.

Abstract

In this paper, we consider a class of difference-of-convex (DC) optimization problems, which require only a weaker restricted -smooth adaptable property on the smooth part of the objective function, instead of the standard global Lipschitz gradient continuity assumption. Such problems are prevalent in many contemporary applications such as compressed sensing, statistical regression, and machine learning, and can be solved by a general Bregman proximal DC algorithm (BPDCA). However, the existing BPDCA is developed based on the stringent requirement that the involved subproblems must be solved exactly, which is often impractical and limits the applicability of the BPDCA. To facilitate the practical implementations and wider applications of the BPDCA, we develop an inexact Bregman proximal difference-of-convex algorithm (iBPDCA) by incorporating two types of relative-type stopping criteria for solving the subproblems. The proposed inexact framework has considerable flexibility to encompass many existing exact and inexact methods, and can accommodate different types of errors that may occur when solving the subproblem. This enables the potential application of our inexact framework across different DC decompositions to facilitate the design of a more efficient DCA scheme in practice. The global subsequential convergence and the global sequential convergence of our iBPDCA are established under suitable conditions including the Kurdyka-Łojasiewicz property. Some numerical experiments are conducted to show the superior performance of our iBPDCA in comparison to existing algorithms. These results also empirically validate the necessity and significance of developing different types of stopping criteria to facilitate the efficient computation of the subproblem in each iteration of our iBPDCA.
Paper Structure (18 sections, 10 theorems, 128 equations, 2 figures, 4 tables, 3 algorithms)

This paper contains 18 sections, 10 theorems, 128 equations, 2 figures, 4 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Notation and preliminaries
  3. An inexact Bregman proximal difference-of-convex algorithm
  4. Convergence analysis
  5. Global convergence of the whole sequence
  6. Numerical experiments
  7. The $\ell_{1-2}$ regularized least squares problem
  8. A dual semi-smooth Newton method for solving the subproblem \ref{['subprob_l12reg']}
  9. Comparison results
  10. The constrained $\ell_{1-2}$ sparse optimization problem
  11. A dual semismooth Newton method for the subproblem \ref{['subprobleml12con']}
  12. Comparison results
  13. Conclusions
  14. Missing proofs and derivations in Section \ref{['sec-num']}
  15. Derivation of the dual problem \ref{['subprobdual_l12reg']}
  16. ...and 3 more sections

Key Result

Proposition 2.1

Suppose that $h: \mathbb{R}^n \rightarrow \mathbb{R} \cup \{+\infty\}$ is a proper closed function and $\Gamma$ is a compact set. If $h \equiv \zeta$ on $\Gamma$ for some constant $\zeta$ and satisfies the KL property at each point of $\Gamma$, then there exist $\varepsilon>0$, $\nu>0$ and $\varphi for all $\bm{x} \in \{\bm{x}\in\mathbb{R}^{n}: \mathrm{dist}(\bm{x},\,\Gamma)<\varepsilon\} \cap \{

Figures (2)

  • Figure 1: Numerical results of GIST, nmAPG, pDCAe, iBPDCA-SC1 and iBPDCA-SC2 for solving the $\ell_{1-2}$ regularized least squares problem on mpg7 and gisette from the UCI data repository.
  • Figure 2: Numerical results of ESQMe, iBPDCA-SC1 and iBPDCA-SC2 for solving the constrained $\ell_{1-2}$ sparse optimization problem on mpg7 and gisette from the UCI data repository.

Theorems & Definitions (23)

  • Definition 2.1: Restricted $L$-smooth adaptable on $\mathcal{X}$
  • Definition 2.2: KL property and KL function
  • Proposition 2.1: Uniformized KL property
  • Lemma 3.1: Approximate sufficient descent property
  • proof
  • Proposition 3.1: Properties of the iBPDCA with (SC1)
  • proof
  • Proposition 3.2: Properties of the iBPDCA with (SC2)
  • proof
  • Theorem 4.1
  • ...and 13 more