Accelerated Gradient Methods via Inertial Systems with Hessian-driven Damping
Zepeng Wang, Juan Peypouquet
TL;DR
The paper develops a unified Lyapunov framework to analyze accelerated gradient methods derived from an inertial system with Hessian-driven damping. By introducing energy-like functions and studying their differential (and discrete) inequalities, the authors derive linear convergence rates under strong convexity, quadratic growth, and Polyak-Łojasiewicz conditions, with explicit dependence on inertial parameters $\alpha,\beta,\gamma$ and time-scaling choices. They discretize the continuous dynamics to obtain gradient and proximal algorithms, showing that these discrete schemes inherit the continuous-time rates and recover established methods such as Nesterov's accelerated gradient and OGM-SC as special cases. The framework provides new insights into the acceleration mechanism, yielding improved rates for P\L and QG settings and offering practical guidance for parameter tuning across gradient and proximal settings. Overall, the work unifies several lineages of accelerated methods under a single Lyapunov-based analysis and clarifies how inertia and Hessian-driven damping influence convergence speed in first-order optimization.
Abstract
We analyze the convergence rate of a family of inertial algorithms, which can be obtained by discretization of an inertial system with Hessian-driven damping. We recover a convergence rate, up to a factor of 2 speedup upon Nesterov's scheme, for smooth strongly convex functions. As a byproduct of our analyses, we also derive linear convergence rates for convex functions satisfying quadratic growth condition or Polyak-Łojasiewicz inequality. As a significant feature of our results, the dependence of the convergence rate on parameters of the inertial system/algorithm is revealed explicitly. This may help one get a better understanding of the acceleration mechanism underlying an inertial algorithm.