Minimizing smooth Kurdyka-Łojasiewicz functions via generalized descent methods: Convergence rate and complexity
Masoud Ahookhosh, Susan Ghaderi, Alireza Kabgani, Morteza Rahimi
TL;DR
This work introduces the generalized descent algorithm (DEAL) for unconstrained minimization of $C^1$ (potentially nonconvex) functions under the Kurdyka–Łojasiewicz (KL) property. By enforcing a generalized descent condition $f(x^{k+1})\le f(x^k)-\rho\|\nabla f(x^k)\|^{\theta}$, DEAL unifies constant step-size, Armijo, proximal-gradient, and high-order proximal-point schemes, and yields global convergence, rate results, and iteration-complexity bounds. A key finding is that linear convergence is achieved when the KL exponent $\vartheta$ and descent order satisfy $\theta=1/\vartheta$, with boosted high-order proximal-point methods attaining global linear convergence for any $\vartheta$ by appropriate regularization order. The analysis provides a simple, broadly applicable KL-based convergence framework that extends to nonsmooth settings via smoothing techniques (e.g., HOME) and demonstrates strong practical performance on inverse problems and LASSO through numerical experiments. Overall, the paper offers a versatile, theoretically grounded toolkit for designing descent algorithms with predictable convergence behavior in KL-structured optimization problems.
Abstract
This paper addresses the generalized descent algorithm (DEAL) for minimizing smooth functions, which is analyzed under the Kurdyka-Łojasiewicz (KL) inequality. In particular, the suggested algorithm guarantees a sufficient decrease by adapting to the cost function's geometry. We leverage the KL property to establish the global convergence, convergence rates, and complexity. A particular focus is placed on the linear convergence of generalized descent methods. We show that the constant step-size and Armijo line search strategies along a generalized descent direction satisfy our generalized descent condition. Additionally, for nonsmooth functions by leveraging the smoothing techniques such as forward-backward and high-order Moreau envelopes, we show that the boosted proximal gradient method (BPGA) and the boosted high-order proximal-point (BPPA) methods are also specific cases of DEAL, respectively. It is notable that if the order of the high-order proximal term is chosen in a certain way (depending on the KL exponent), then the sequence generated by BPPA converges linearly for an arbitrary KL exponent. Our preliminary numerical experiments on inverse problems and LASSO demonstrate the efficiency of the proposed methods, validating our theoretical findings.