Unifying restart accelerated gradient and proximal bundle methods
Jiaming Liang
TL;DR
This paper addresses efficient optimization for convex smooth composite problems by introducing a restarted accelerated gradient method (restart ACG) and situating it within the accelerated inexact proximal point (A-HPE) framework. It also shows that the modern proximal bundle method (MPB) is an instance of the HPE framework, thereby unifying restart ACG and MPB under a single proximal-point perspective. The main contributions are: (i) a restart ACG with optimal iteration complexity for CSCO, (ii) a demonstration that MPB fits the HPE framework as a restarted cutting-plane approach, and (iii) a coherent interpretation of both methods as multi-step proximal-point schemes. This unified view explains the strong practical performance of these restarted schemes and suggests avenues for μ-universal extensions and strong-convexity settings.
Abstract
This paper presents a novel restarted version of Nesterov's accelerated gradient method and establishes its optimal iteration-complexity for solving convex smooth composite optimization problems. The proposed restart accelerated gradient method is shown to be a specific instance of the accelerated inexact proximal point framework introduced in "An accelerated hybrid proximal extragradient method for convex optimization and its implications to second-order methods" by Monteiro and Svaiter, SIAM Journal on Optimization, 2013. Furthermore, this work examines the proximal bundle method within the inexact proximal point framework, demonstrating that it is an instance of the framework. Notably, this paper provides new insights into the underlying algorithmic principle that unifies two seemingly disparate optimization methods, namely, the restart accelerated gradient and the proximal bundle methods.