Tikhonov regularized second-order dynamics with Hessian-driven damping for solving convex optimization problems
Xiangkai Sun, Guoxiang Tian, Huan Zhang
TL;DR
This work introduces a Tikhonov-regularized second-order dynamical system with time scaling and Hessian-driven damping to solve convex problems in a Hilbert space. The authors prove fast convergence of the objective along trajectories, weak convergence to minimizers, and, under appropriate parameter schedules, simultaneous fast function-value decay and strong convergence to the minimum-norm solution; they further derive an inertial proximal gradient algorithm by discretization with matching convergence properties. Theoretical results are complemented by numerical experiments showing improved energy/iteration metrics and favorable comparisons to related Hessian-damping schemes. Overall, the paper offers a cohesive continuous-discrete framework that yields accelerated convergence while stabilizing trajectories in convex optimization.
Abstract
This paper deals with a Tikhonov regularized second-order dynamical system that incorporates time scaling, asymptotically vanishing damping and Hessian-driven damping for solving convex optimization problems. Under appropriate setting of the parameters, we first obtain fast convergence results of the function value along the trajectory generated by the dynamical system. Then, we show that the trajectory generated by the dynamical system converges weakly to a minimizer of the convex optimization problem. We also demonstrate that, by properly tuning these parameters, both fast convergence rates of the function value and strong convergence of the trajectory towards the minimum norm solution of the convex optimization problem can be achieved simultaneously. Furthermore, we study convergence properties of an inertial proximal gradient algorithm obtained by the temporal discretization of the dynamical system. Finally, we present numerical experiments to illustrate the obtained results.