Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the convergence rate of the boosted Difference-of-Convex Algorithm (DCA)

Hadi Abbaszadehpeivasti, Etienne de Klerk, Adrien Taylor

TL;DR

The paper analyzes the unconstrained difference-of-convex (DC) optimization problem using boosted DCA and provides a worst-case convergence bound that improves over standard DCA when a positive boost is applied. It employs a semidefinite programming performance-estimation (PEP) framework to bound the worst-case behavior over DC decompositions with bounded curvature, and formalizes the analysis through three proof stages (reformulation, single-iteration reduction, and single-iteration analysis). The main result shows a decrease in the gradient-gap measure at a rate depending on the curvature ratio $κ=μ/L$ and boost parameter $α$, with $0≤α≤\min\{1,2κ\}$, recovering the DCA bound when $α=0$ and potentially yielding strictly faster convergence for $κ>0$. The paper further connects boosted DCA to gradient descent under the Polyak–Łojasiewicz (PL) condition, deriving a linear convergence bound with an optimal boosted step length $\frac{1+α^*}{L}$ and an explicit improvement over prior bounds for small $κ$, thereby providing a theoretical explanation for empirical improvements observed with boosting. Overall, it extends SDP-based performance analysis to boosted DC methods and offers a principled framework for understanding when boosting helps, with practical implications for step-length choices and algorithm design.

Abstract

The difference-of-convex algorithm (DCA) is a well-established nonlinear programming technique that solves successive convex optimization problems. These sub-problems are obtained from the difference-of-convex~(DC) decompositions of the objective and constraint functions. We investigate the worst-case performance of the unconstrained DCA, with and without boosting, where boosting simply performs an additional step in the direction generated by the usual DCA method. We show that, for certain classes of DC decompositions, the boosted DCA is provably better in the worst-case than the usual DCA. While several numerical studies have reported that boosted DCA outperforms classical DCA, a theoretical explanation for this behavior has, to the best of our knowledge, not been given until now. Our proof technique relies on semidefinite programming (SDP) performance estimation.

On the convergence rate of the boosted Difference-of-Convex Algorithm (DCA)

TL;DR

The paper analyzes the unconstrained difference-of-convex (DC) optimization problem using boosted DCA and provides a worst-case convergence bound that improves over standard DCA when a positive boost is applied. It employs a semidefinite programming performance-estimation (PEP) framework to bound the worst-case behavior over DC decompositions with bounded curvature, and formalizes the analysis through three proof stages (reformulation, single-iteration reduction, and single-iteration analysis). The main result shows a decrease in the gradient-gap measure at a rate depending on the curvature ratio and boost parameter , with , recovering the DCA bound when and potentially yielding strictly faster convergence for . The paper further connects boosted DCA to gradient descent under the Polyak–Łojasiewicz (PL) condition, deriving a linear convergence bound with an optimal boosted step length and an explicit improvement over prior bounds for small , thereby providing a theoretical explanation for empirical improvements observed with boosting. Overall, it extends SDP-based performance analysis to boosted DC methods and offers a principled framework for understanding when boosting helps, with practical implications for step-length choices and algorithm design.

Abstract

The difference-of-convex algorithm (DCA) is a well-established nonlinear programming technique that solves successive convex optimization problems. These sub-problems are obtained from the difference-of-convex~(DC) decompositions of the objective and constraint functions. We investigate the worst-case performance of the unconstrained DCA, with and without boosting, where boosting simply performs an additional step in the direction generated by the usual DCA method. We show that, for certain classes of DC decompositions, the boosted DCA is provably better in the worst-case than the usual DCA. While several numerical studies have reported that boosted DCA outperforms classical DCA, a theoretical explanation for this behavior has, to the best of our knowledge, not been given until now. Our proof technique relies on semidefinite programming (SDP) performance estimation.
Paper Structure (8 sections, 6 theorems, 38 equations, 4 figures, 1 table, 2 algorithms)

This paper contains 8 sections, 6 theorems, 38 equations, 4 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Performance estimation
  3. Sublinear convergence of the boosted DCA
  4. Part 1 of the proof: reformulation.
  5. Part 2 of the proof: reducing to single iteration analysis.
  6. Part 3 of the proof: performing the single iteration analysis.
  7. Consequences for gradient descent
  8. Conclusion and discussion

Key Result

Lemma 1

Assume that $f$ is continuously differentiable, and $f_1$ and $f_2$ have minimum curvature $\mu > 0$. Then, with reference to Algorithm Alg:BCDA, one has

Figures (4)

  • Figure 1: Performance bounds for DCA ($\alpha=0$) and boosted DCA when $L_1=L_2=2$, and $\mu_1=1,\ \mu_2=0$.
  • Figure 2: Performance bounds for DCA ($\alpha=0$) and boosted DCA when $L_1=L_2=2$, and $\mu_1=0,\ \mu_2=1$.
  • Figure 3: Performance bounds for DCA ($\alpha=0$) and boosted DCA for the case that $L_1=1\ L_2=2$, and $\mu_1=0.5,\ \mu_2=0.25$ and $N=10$.
  • Figure 4: Comparisons between the bounds from Corollary \ref{['cor:GD PL opt']} on $\frac{f(x^2)-f^\star}{f(x^1)-f^\star}$ for gradient descent with step-size $\frac{3}{(2+\kappa ) L}$ for a range of $\kappa$. The 'numerically computed improvement' uses the stronger characterization of smooth Łojasiewicz functions from rubbens2025constructive.

Theorems & Definitions (9)

  • Lemma 1: Proposition 4 in Boosted_DCA_smooth_MP
  • Theorem 1
  • Theorem 2
  • Lemma 2: abbaszad2024DCA, Lemma 2.1 (Descent lemma)
  • Definition 1
  • proof
  • Theorem 3
  • proof
  • Corollary 1