Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Iteration complexity of the Difference-of-Convex Algorithm for unconstrained optimization: a simple proof

Serge Gratton, Philippe L. Toint

TL;DR

The paper analyzes the global iteration complexity of the DC Algorithm (DCA) for unconstrained minimization of $f(x)=g(x)-h(x)$, where $g$ is μ-strongly convex and ∇h is Lipschitz with constant $L_h$. It provides a simple, explicit proof that the average squared gradient norms along iterates satisfy an upper bound $\frac{1}{\lfloor k/2 \rfloor}\sum_{i=\lceil k/2 \rceil}^k \|\nabla f(x_i)\|^2 = o(1/k)$, and it constructs a univariate slow-convergence example with $g(x)=\tfrac12 x^2$ and a convex $h$ with Lipschitz gradient yielding $\|\nabla f(x_k)\|^2 = 1/(k+1)^{1+2\delta}$ for any $\delta>0$. These results show that, in the worst case, DCA does not improve over steepest descent in iteration complexity, even though practical performance may benefit from exploiting DC structure or algorithmic variants. The work clarifies fundamental limits of DC-based optimization for iteration efficiency and informs when DC structure may or may not provide convergence advantages.

Abstract

We propose a simple proof of the worst-case iteration complexity for the Difference of Convex functions Algorithm (DCA) for unconstrained minimization, showing that the global rate of convergence of the norm of the objective function's gradients at the iterates converge to zero like o(1/k). A small example is also provided indicating that the rate cannot be improved.

Iteration complexity of the Difference-of-Convex Algorithm for unconstrained optimization: a simple proof

TL;DR

The paper analyzes the global iteration complexity of the DC Algorithm (DCA) for unconstrained minimization of , where is μ-strongly convex and ∇h is Lipschitz with constant . It provides a simple, explicit proof that the average squared gradient norms along iterates satisfy an upper bound , and it constructs a univariate slow-convergence example with and a convex with Lipschitz gradient yielding for any . These results show that, in the worst case, DCA does not improve over steepest descent in iteration complexity, even though practical performance may benefit from exploiting DC structure or algorithmic variants. The work clarifies fundamental limits of DC-based optimization for iteration efficiency and informs when DC structure may or may not provide convergence advantages.

Abstract

We propose a simple proof of the worst-case iteration complexity for the Difference of Convex functions Algorithm (DCA) for unconstrained minimization, showing that the global rate of convergence of the norm of the objective function's gradients at the iterates converge to zero like o(1/k). A small example is also provided indicating that the rate cannot be improved.
Paper Structure (3 sections, 2 theorems, 7 equations, 1 figure)

This paper contains 3 sections, 2 theorems, 7 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Global rate of convergence
  3. Discussion

Key Result

Theorem 2.1

Consider applying the DCA to problem (problem). Suppose that $g$ is continuously differentiable and strongly convex with constant $\mu>0$, that is and that $h$ is convex, continuously differentiable and that its gradient is Lipschitz continuous with constant $L_h$, that is Suppose finally that there exists $f_{\rm low}\in \hbox{I R}$ such that $f(x)\ge f_{\rm low}$ for all $x\in \hbox{I R}^n$.

Figures (1)

  • Figure 1: The shapes of $f$, $g$ and $h$ on $[x_{25},x_0]$

Theorems & Definitions (2)

  • Theorem 2.1
  • Theorem 2.2