Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A preconditioned second-order convex splitting algorithm with extrapolation

Xinhua Shen, Hongpeng Sun

TL;DR

The paper tackles nonconvex optimization by integrating a preconditioned, second-order convex-splitting scheme with extrapolation, combining BDF2 in the implicit part and Adams–Bashforth in the explicit part while introducing a preconditioned proximal term. It establishes global convergence under Kurdyka–Łojasiewicz properties and demonstrates numerical efficiency through LC/SCAD least-squares and graph-based Ginzburg–Landau segmentation experiments. The key contributions are a robust algorithmic framework with extrapolation, a unified convergence theory, and practical validation showing reduced computation times. This approach offers a scalable, theoretically sound method for large-scale nonconvex problems with potential extensions like line-search variants.

Abstract

Nonconvex optimization problems are widespread in modern machine learning and data science. We introduce an extrapolation strategy into a class of preconditioned second-order convex splitting algorithms for nonconvex optimization problems. The proposed algorithms combine second-order backward differentiation formulas (BDF2) with an extrapolation method. Meanwhile, the implicit-explicit scheme simplifies the subproblem through a preconditioned process. As a result, our approach solves nonconvex problems efficiently without significant computational overhead. Theoretical analysis establishes global convergence of the algorithms using Kurdyka-Łojasiewicz properties. Numerical experiments include a benchmark problem, the least squares problem with SCAD regularization, and an image segmentation problem. These results demonstrate that our algorithms are highly efficient, as they achieve reduced solution times and competitive performance.

A preconditioned second-order convex splitting algorithm with extrapolation

TL;DR

The paper tackles nonconvex optimization by integrating a preconditioned, second-order convex-splitting scheme with extrapolation, combining BDF2 in the implicit part and Adams–Bashforth in the explicit part while introducing a preconditioned proximal term. It establishes global convergence under Kurdyka–Łojasiewicz properties and demonstrates numerical efficiency through LC/SCAD least-squares and graph-based Ginzburg–Landau segmentation experiments. The key contributions are a robust algorithmic framework with extrapolation, a unified convergence theory, and practical validation showing reduced computation times. This approach offers a scalable, theoretically sound method for large-scale nonconvex problems with potential extensions like line-search variants.

Abstract

Nonconvex optimization problems are widespread in modern machine learning and data science. We introduce an extrapolation strategy into a class of preconditioned second-order convex splitting algorithms for nonconvex optimization problems. The proposed algorithms combine second-order backward differentiation formulas (BDF2) with an extrapolation method. Meanwhile, the implicit-explicit scheme simplifies the subproblem through a preconditioned process. As a result, our approach solves nonconvex problems efficiently without significant computational overhead. Theoretical analysis establishes global convergence of the algorithms using Kurdyka-Łojasiewicz properties. Numerical experiments include a benchmark problem, the least squares problem with SCAD regularization, and an image segmentation problem. These results demonstrate that our algorithms are highly efficient, as they achieve reduced solution times and competitive performance.

Paper Structure

This paper contains 14 sections, 10 theorems, 70 equations, 4 figures, 5 tables, 3 algorithms.

Table of Contents

  1. Key words.
  2. MSCodes.
  3. Introduction
  4. Preliminary
  5. Algorithm formulation and convergence analysis
  6. Preconditioned second-order backward differential formula algorithm with extrapolation
  7. Global subsequence convergence
  8. Convergence results
  9. The unified algorithm framework
  10. Preconditioning method and numerical experiment
  11. Preconditioned technique
  12. Least squares problem with SCAD regularizer
  13. Segmentation problem by the graphic Ginzburg-Landau model
  14. Conclusion

Key Result

Lemma 1

Let $L>0$ be the Lipschitz constant of $f(u)$. If then the function $F^n(\cdot)$ is convex. Moreover, if $\delta t$ is strictly less than $\frac{1}{2L}$, $F^n(\cdot)$ becomes strongly convex with a modulus of $\frac{1}{2\delta t} - L$.

Figures (4)

  • Figure 1: The index '$\text{res}(u^k)$' performance
  • Figure 2: The convergence rate
  • Figure 3: The performance of segmentation assignment.
  • Figure 4: The performance of segmentation assignment.

Theorems & Definitions (26)

  • Definition 1: KL property, KL function ABS Artacho2018 and KL exponent Bolte2014
  • Lemma 1
  • proof
  • Proposition 1
  • proof
  • Remark 1
  • Lemma 2
  • proof
  • Proposition 2
  • proof
  • ...and 16 more