Recovering Nesterov accelerated dynamics from Heavy Ball dynamics via time rescaling
Hedy Attouch, Radu Ioan Bot, David Alexander Hulett, Dang-Khoa Nguyen
TL;DR
The paper reveals that two foundational second-order dynamics—the Heavy Ball with friction and the Su–Boyd–Candès accelerated gradient—are connected through time reparametrizations in both convex optimization and monotone inclusion settings. By constructing appropriate time-scaling functions, the authors show that convergence properties and rates for one dynamics can be transferred to the other, enabling a unified view of accelerated methods. They extend these ideas to operator settings, introducing inertial corrections and deriving conditions under which residuals vanish and trajectories converge weakly to zeros of maximally monotone operators, with explicit links to Fast OGDA. The results provide a principled mechanism to translate continuous-time acceleration results across optimization and monotone framework, and they yield explicit rate estimates under natural growth and Lipschitz assumptions, enhancing understanding of acceleration phenomena in both single-valued and set-valued contexts.
Abstract
In a real Hilbert space, we consider two classical problems: the global minimization of a smooth and convex function $f$ (i.e., a convex optimization problem) and finding the zeros of a monotone and continuous operator $V$ (i.e., a monotone equation). Attached to the optimization problem, first we study the asymptotic properties of the trajectories generated by a second-order dynamical system which features a constant viscous friction coefficient and a positive, monotonically increasing function $b(\cdot)$ multiplying $\nabla f$. For a generated solution trajectory $y(t)$, we show small $o$ convergence rates dependent on $b(t)$ for $f(y(t)) - \min f$, and the weak convergence of $y(t)$ towards a global minimizer of $f$. In 2015, Su, Boyd and Candés introduced a second-order system which could be seen as the continuous-time counterpart of Nesterov's accelerated gradient. As the first key point of this paper, we show that for a special choice for $b(t)$, these two seemingly unrelated dynamical systems are connected: namely, they are time reparametrizations of each other. Every statement regarding the continuous-time accelerated gradient system may be recovered from its Heavy Ball counterpart. As the second key point of this paper, we observe that this connection extends beyond the optimization setting. Attached to the monotone equation involving the operator $V$, we again consider a Heavy Ball-like system which features an additional correction term which is the time derivative of the operator along the trajectory. We establish a time reparametrization equivalence with the Fast OGDA dynamics introduced by Bot, Csetnek and Nguyen in 2022, which can be seen as an analog of the continuous accelerated gradient dynamics, but for monotone operators. Again, every statement regarding the Fast OGDA system may be recovered from a Heavy Ball-like system.