A Modified Proximal Bundle Method Under A Frank-Wolfe Perspective
David Fersztand, Xu Andy Sun
TL;DR
This work investigates MPB-FA, a modified proximal bundle method with a fixed absolute-accuracy null-step test and a fixed proximal parameter, and reveals a dual relationship to a Fully Corrective Frank–Wolfe algorithm on the Moreau envelope of the dual problem. By leveraging this dual perspective, the authors extend linear convergence results to general piecewise-linear f and derive an improved iteration complexity for MPB-FA, achieving near-optimal rates of order $ ilde{O}(\epsilon^{-4/5})$ (up to logarithmic factors) in the $L$-smooth setting. They provide a two-tier analysis: an initial convergence-rate bound, followed by an enhanced rate that accounts for different categories of serious steps (consecutive, distant-center, small-norm), along with insights into effective bundle management. The paper also frames MPB-FA as an inexact augmented Lagrangian method in the dual, solved via an approximate FCW, and offers numerical experiments that support active-cut bundle management as advantageous for practical performance. Overall, the results advance the theory of proximal-bundle methods for nonsmooth convex optimization, linking them to projection-free Frank–Wolfe techniques and delivering improved complexity guarantees for a broad class of problems.
Abstract
The proximal bundle method (PBM) is a fundamental and computationally effective algorithm for solving optimization problems with nonsmooth components. In this paper, we conduct a theoretical investigation of a modified proximal bundle method, which we call the Modified Proximal Bundle with Fixed Absolute Accuracy (MPB-FA). MPB-FA modifies PBM in two key aspects. Firstly, the null-step test of MPB-FA is based on an absolute accuracy criterion, and the accuracy is fixed over iterations, while the standard PBM uses a relative accuracy in the null-step test, which changes with iterations. Secondly, the proximal parameter in MPB-FA is also fixed over iterations, while it is permitted to change in the standard PBM. These modifications allow us to interpret a sequence of null steps of MPB-FA as a Frank-Wolfe algorithm on the Moreau envelope of the dual problem. In light of this correspondence, we first extend the linear convergence of Kelley's method on convex piecewise linear functions from the positive homogeneous to the general case. Building on this result, we propose a novel complexity analysis of MPB-FA and derive an $\mathcal{O}(ε^{-4/5})$ iteration complexity, improving upon the best known $\mathcal{O}(ε^{-2})$ guarantee on a related variant of PBM. It is worth-noting that the best known complexity bound for the classic PBM is $\mathcal{O}(ε^{-3})$. Our approach also reveals new insights on bundle management.