Inertial dynamical systems and accelerated algorithms with implicit Hessian-driven damping for nonconvex optimization
Zeying Gao, Xiangkai Sun, Liang He
TL;DR
This work addresses nonconvex optimization where the objective $f$ is strongly quasiconvex and typically lacks Lipschitz-continuous gradients. It introduces an inertial dynamical system with implicit Hessian-driven damping, proving exponential convergence of both the trajectory and objective gap via a Lyapunov analysis. By explicit time discretization, the authors derive an inertial accelerated algorithm and a perturbed version, establishing linear convergence under suitable parameter choices and, for perturbations decaying as $O(k^{-p})$, accelerated decay rates $O(k^{-2p})$ for the objective and $O(k^{-p})$ for the iterates. Numerical experiments corroborate the theory, showing faster convergence and less oscillation than existing Heavy Ball and Hessian-enhanced methods, highlighting the practical impact for strongly quasiconvex nonconvex problems. The methods extend acceleration techniques to a broader nonconvex setting and provide robust performance without requiring gradient Lipschitz continuity.
Abstract
This paper is devoted to the investigation of inertial dynamical systems with implicit Hessian-driven damping for strongly quasiconvex optimization which is a specific class of nonconvex optimization problems. We first establish exponential convergence rate properties for this system without requiring Lipschitz continuity of the gradient on the function. Then, we obtain an inertial accelerated algorithm for minimizing strongly quasiconvex functions through natural explicit time discretization to the dynamical system. Meanwhile, we consider an exogenous additive perturbation term to this dynamical system and obtain the corresponding algorithm. By utilizing the Lyapunov method, we establish convergence rates of iterative sequences and their function values. Furthermore, we conduct numerical experiments to illustrate the theoretical results.