Convergence analysis of a proximal-type algorithm for DC programs with applications to variable selection
Shuang Wu, Bui Van Dinh, Liguo Jiao, Do Sang Kim, Wensheng Zhu
TL;DR
It is shown that the point computed by proximal point algorithm at each iteration can be used to determine a descent direction for the objective function at this point and this algorithm can be considered as a combination of proximal point algorithm together with a linesearch step that uses this descent direction.
Abstract
We consider a minimization problem of the form $P(\varphi, g, h):$ $$\min\left\{f(x):= \varphi(x) + g(x) - h(x) \colon x \in \mathbb{R}^n\right\},$$ where $\varphi$ is a differentiable function and $g,$ $h$ are convex functions, and introduce iterative methods to finding a critical point of $f$ when $f$ is differentiable. We show that the point computed by proximal point algorithm at each iteration can be used to determine a descent direction for the objective function at this point. This algorithm can be considered as a combination of proximal point algorithm together with a linesearch step that uses this descent direction. We also study convergence results of these algorithms and the inertial proximal methods proposed by Maing$\acute{e}$ and Moudafi (SIAM J. Optim. {\bf 19}(2008), 397--413) under the main assumption that the objective function satisfies the Kurdika--Łojasiewicz property. The proposed algorithm is then applied to solve the variable selection problem in linear regression.