Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A boosted second-order convex splitting algorithm based on gradient flows

Xinhua Shen, Zaijiu Shang, Hongpeng Sun

TL;DR

The paper tackles nonconvex optimization with a DC structure by introducing a preconditioned, second-order convex-splitting algorithm that couples BDF2 for the implicit part with Adams-Bashforth for the explicit part and augments it with an Armijo-type line search. The authors prove global convergence using Kurdyka–Łojasiewicz theory under mild assumptions and demonstrate that preconditioning and line search dramatically improve efficiency over standard DC methods on large-scale problems. They validate the approach with numerical experiments on SCAD-regularized least squares and a graphic Ginzburg–Landau segmentation model, showing faster convergence and favorable objective values. The work highlights the practical impact of combining dynamic DC framing, second-order IMEX discretization, line search, and preconditioning for scalable nonconvex optimization.

Abstract

This paper introduces a preconditioned convex splitting algorithm enhanced by line search techniques for nonconvex optimization problems. The algorithm utilizes second-order backward differentiation formulas (BDF) for the implicit components and employs the Adams-Bashforth scheme for the nonlinear and explicit parts of the gradient flow in variational problems. The resulting scheme can be interpreted as a varying (or dynamic) difference-of-convex (DC) algorithm. It integrates the Armijo line search strategy to improve performance. The study also discusses classical preconditioners such as symmetric Gauss-Seidel and Jacobi within this context. The global convergence of the algorithm is established through the Kurdyka-Łojasiewicz properties, ensuring convergence under the preconditioned scheme. Numerical experiments demonstrate significantly higher efficiency than standard DC algorithms and other boosted algorithms.

A boosted second-order convex splitting algorithm based on gradient flows

TL;DR

The paper tackles nonconvex optimization with a DC structure by introducing a preconditioned, second-order convex-splitting algorithm that couples BDF2 for the implicit part with Adams-Bashforth for the explicit part and augments it with an Armijo-type line search. The authors prove global convergence using Kurdyka–Łojasiewicz theory under mild assumptions and demonstrate that preconditioning and line search dramatically improve efficiency over standard DC methods on large-scale problems. They validate the approach with numerical experiments on SCAD-regularized least squares and a graphic Ginzburg–Landau segmentation model, showing faster convergence and favorable objective values. The work highlights the practical impact of combining dynamic DC framing, second-order IMEX discretization, line search, and preconditioning for scalable nonconvex optimization.

Abstract

This paper introduces a preconditioned convex splitting algorithm enhanced by line search techniques for nonconvex optimization problems. The algorithm utilizes second-order backward differentiation formulas (BDF) for the implicit components and employs the Adams-Bashforth scheme for the nonlinear and explicit parts of the gradient flow in variational problems. The resulting scheme can be interpreted as a varying (or dynamic) difference-of-convex (DC) algorithm. It integrates the Armijo line search strategy to improve performance. The study also discusses classical preconditioners such as symmetric Gauss-Seidel and Jacobi within this context. The global convergence of the algorithm is established through the Kurdyka-Łojasiewicz properties, ensuring convergence under the preconditioned scheme. Numerical experiments demonstrate significantly higher efficiency than standard DC algorithms and other boosted algorithms.

Paper Structure

This paper contains 17 sections, 22 theorems, 142 equations, 3 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Key words.
  2. MSCodes.
  3. Introduction
  4. Algorithm and convergence analysis
  5. Preliminary
  6. Algorithm and energy decay
  7. Global convergence of for the case of $\hat{u}^n = u^n$.
  8. Global convergence for the case of $\hat{u}^n = \tilde{u}^n$
  9. Convergence analysis for the special case with nonsmooth $H(u)$
  10. Some discussions on the KL property assumption
  11. Preconditioners and numerical experiments
  12. The preconditioning techniques along with the order of preconditioned scheme
  13. Numerical experiments
  14. Least squares problems with SCAD and modified SCAD regularizer
  15. Graphic Ginzburg-Landau model
  16. ...and 2 more sections

Key Result

Lemma 1

Suppose that the Lipschitz constant of $f(u)$ is $L > 0$. When $\delta t < \frac{2}{3L}$, $F^n(u)$ is strongly convex.

Figures (3)

  • Figure 1: Convergence performance of different optimization algorithms ($P_1(u) = \mu\|u\|_1$ (L1-) and $P_1(u) = \mu\mathcal{H}_{\alpha}(u)$ (H-))
  • Figure 2: The true solution and the solution obtained by \ref{['eq:lsp']} with $P_1(x)=\mu\|x\|_1$ and $\mu=0.03$
  • Figure 3: Segmentation results for the stone image

Theorems & Definitions (47)

  • Definition 1: KL property, KL function and KL exponent
  • Lemma 1
  • proof
  • Proposition 1
  • proof
  • Proposition 2
  • Remark 1
  • Proposition 3
  • proof
  • Lemma 2
  • ...and 37 more