Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Contractive difference-of-convex algorithms

Songnian He, Qiao-Li Dong, Michael Th. Rassias

TL;DR

The paper tackles nonconvex DC optimization by proposing the contractive difference-of-convex algorithm (cDCA), which reframes the subproblem in the DCA framework as a fixed-point problem of a contraction and solves it via adaptive Picard iterations. It establishes global subsequential convergence and, under standard assumptions including the Kurdyka--Łojasiewicz property, global convergence of the whole sequence to a critical point, with potential convergence-rate insights when the KL exponent is of a power form. The approach yields practical benefits, providing an adaptive inner termination rule that ensures progress without excessive inner iterations, and theoretical guarantees for convergence. Numerical experiments on DC-regularized least-squares problems demonstrate that cDCA often requires fewer iterations and less CPU time than ADCA and pDCA$_e$, while achieving comparable objective values, indicating meaningful efficiency gains in DC optimization tasks.

Abstract

The difference-of-convex algorithm (DCA) and its variants are the most popular methods to solve the difference-of-convex optimization problem. Each iteration of them is reduced to a convex optimization problem, which generally needs to be solved by iterative methods such as proximal gradient algorithm. However, these algorithms essentially belong to some iterative methods of fixed point problems of averaged mappings, and their convergence speed is generally slow. Furthermore, there is seldom research on the termination rule of these iterative algorithms solving the subproblem of DCA. To overcome these defects, we ffrstly show that the subproblem of the linearized proximal method (LPM) in each iteration is equal to the ffxed point problem of a contraction. Secondly, by using Picard iteration to approximately solve the subproblem of LPM in each iteration, we propose a contractive difference-ofconvex algorithm (cDCA) where an adaptive termination rule is presented. Both global subsequential convergence and global convergence of the whole sequence of cDCA are established. Finally, preliminary results from numerical experiments are promising.

Contractive difference-of-convex algorithms

TL;DR

The paper tackles nonconvex DC optimization by proposing the contractive difference-of-convex algorithm (cDCA), which reframes the subproblem in the DCA framework as a fixed-point problem of a contraction and solves it via adaptive Picard iterations. It establishes global subsequential convergence and, under standard assumptions including the Kurdyka--Łojasiewicz property, global convergence of the whole sequence to a critical point, with potential convergence-rate insights when the KL exponent is of a power form. The approach yields practical benefits, providing an adaptive inner termination rule that ensures progress without excessive inner iterations, and theoretical guarantees for convergence. Numerical experiments on DC-regularized least-squares problems demonstrate that cDCA often requires fewer iterations and less CPU time than ADCA and pDCA, while achieving comparable objective values, indicating meaningful efficiency gains in DC optimization tasks.

Abstract

The difference-of-convex algorithm (DCA) and its variants are the most popular methods to solve the difference-of-convex optimization problem. Each iteration of them is reduced to a convex optimization problem, which generally needs to be solved by iterative methods such as proximal gradient algorithm. However, these algorithms essentially belong to some iterative methods of fixed point problems of averaged mappings, and their convergence speed is generally slow. Furthermore, there is seldom research on the termination rule of these iterative algorithms solving the subproblem of DCA. To overcome these defects, we ffrstly show that the subproblem of the linearized proximal method (LPM) in each iteration is equal to the ffxed point problem of a contraction. Secondly, by using Picard iteration to approximately solve the subproblem of LPM in each iteration, we propose a contractive difference-ofconvex algorithm (cDCA) where an adaptive termination rule is presented. Both global subsequential convergence and global convergence of the whole sequence of cDCA are established. Finally, preliminary results from numerical experiments are promising.
Paper Structure (10 sections, 10 theorems, 57 equations, 2 figures, 4 tables, 2 algorithms)

This paper contains 10 sections, 10 theorems, 57 equations, 2 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. The contractive difference-of-convex algorithm
  4. Motivation
  5. Algorithm
  6. Convergence analysis
  7. Numerical Experiments
  8. Optimal choice of the parameter $\lambda$
  9. Comparison with ADCA and pDCA$_e$
  10. Concluding Remarks

Key Result

Lemma 2.1

(XuHK) Let $T: \mathbb{R}^n\rightarrow \mathbb{R}^n$ be $\beta$-cocoercive. Then for any $\nu\in (0, 2\beta]$, $I-\nu T$ is a nonexpansive mapping.

Figures (2)

  • Figure 1: The true solution and the solution by cDCA for $\ell_{1-2}$ regularizer (left) and logarithmic regularizer (right).
  • Figure 2: The decay of objective values of cDCA, ADCA and pDCA$_e$ with CPU time for $\ell_{1-2}$ regularizer (left) and logarithmic regularizer (right).

Theorems & Definitions (20)

  • Lemma 2.1
  • Lemma 2.2
  • Lemma 2.3
  • Lemma 2.4
  • Lemma 2.5
  • Definition 2.1
  • Definition 3.1
  • Lemma 3.1
  • proof
  • Remark 3.1
  • ...and 10 more