On the Acceleration of Proximal Bundle Methods

TL;DR

This work addresses accelerating the proximal bundle method (PBM) for convex optimization by recasting PBM as an inexact proximal-point method and introducing an accelerated scheme that leverages a smooth lower interpolating model. The key innovations include a novel null-step test and an oracle that replaces cutting planes with differentiable lower models computable via convex QCQPs, enabling an overall iteration complexity of $O\left(\frac{1}{\sqrt{ε}}\log\left(\frac{1}{ε}\right)\right)$ for smooth objectives. The authors extend the acceleration to composite problems, where the objective is a sum of a smooth component and a piecewise linear one, achieving the same rate up to dimension-dependent factors. These results answer open questions about PBM acceleration, connect the method to accelerated proximal-point theory, and provide practical insight for bundle management and adaptive proximal parameters. The approach thus broadens the applicability and efficiency of bundle methods in both smooth and nonsmooth convex settings.

Abstract

The proximal bundle method (PBM) is a fundamental and computationally effective algorithm for solving nonsmooth optimization problems. In this paper, we present the first variant of the PBM for smooth objectives, achieving an accelerated convergence rate of $O(ε^{-1/2}\log(1/ε))$, where $ε$ is the desired accuracy. Our approach addresses an open question regarding the convergence guarantee of proximal bundle type methods, which was previously posed in two recent papers. We interpret the PBM as a proximal point algorithm and base our proposed algorithm on an accelerated inexact proximal point scheme. Our variant introduces a novel null step test and oracle while maintaining the core structure of the original algorithm. The newly proposed oracle substitutes the traditional cutting planes with a smooth lower approximation of the true function. We show that this smooth interpolating lower model can be computed as a convex quadratic program. We also examine a second setting where Nesterov acceleration can be effectively applied, specifically when the objective is the sum of a smooth function and a piecewise linear one.

Theorems & Definitions (60)

• Remark 1: Extending the algorithm to a composite setting
• Definition 2
• Remark 3: Comparison with the recent approach by Florea and Nesterov florea2024optimallowerboundsmooth
• Lemma 4
• proof
• Definition 5: Convex hull of a collection of functions
• Lemma 6
• proof
• Theorem 7
• proof