A preconditioned difference of convex functions algorithm with extrapolation and line search
Ran Zhang, Hongpeng Sun
TL;DR
This work develops two preconditioned proximal DC algorithms that integrate extrapolation with an aggressive non-monotone line search to tackle nonconvex optimization of the form $E(x)=f(x)+g_1(x)-g_2(x)$. The extrapolation parameter is adaptively governed by a line search, and a preconditioner $M$ facilitates efficient proximal updates. Global convergence is established under Kurdyka–Łojasiewicz properties for the augmented energy functions, with convergence rates characterized by the KL exponent. Numerical experiments on SCAD-regularized least squares and nonlocal Ginzburg–Landau image segmentation show substantial improvements in convergence speed and solution accuracy, including GPU-accelerated performance. The framework maintains the simplicity of DC methods while delivering robust acceleration and theoretical guarantees for a broad class of nonconvex problems.
Abstract
This paper proposes a novel proximal difference-of-convex (DC) algorithm enhanced with extrapolation and aggressive non-monotone line search for solving non-convex optimization problems. We introduce an adaptive conservative update strategy of the extrapolation parameter determined by a computationally efficient non-monotone line search. The core of our algorithm is to unite the update of the extrapolation parameter with the step size of the non-monotone line search interactively. The global convergence of the two proposed algorithms is established through the Kurdyka-Łojasiewicz properties, ensuring convergence within a preconditioned framework for linear equations. Numerical experiments on two general non-convex problems: SCAD-penalized binary classification and graph-based Ginzburg-Landau image segmentation models, demonstrate the proposed method's high efficiency compared to existing DC algorithms both in convergence rate and solution accuracy.