Universal subgradient and proximal bundle methods for convex and strongly convex hybrid composite optimization

TL;DR

The paper tackles convex hybrid composite optimization by introducing two parameter-free, $\mu_\phi$-universal methods, U-CS and U-PB, for solving $\phi(x)=f(x)+h(x)$ without knowledge of problem constants or the optimal value. Both methods are analyzed within a unified FSCO framework, which yields functional oracle-complexity bounds that scale with the intrinsic strong convexity parameter $\mu_\phi$ and, when applicable, with $\mu_h$. U-CS pertains to a universal composite subgradient approach with a line-search on the prox-stepsize and $\chi$-dependent universality, while U-PB extends proximal bundle methods with adaptive stepsizes and bundle updates; notably, U-CS is a special case of U-PB with $\overline N=1$. The results show near-optimal/tilde-optimal rates in key regimes and establish complete universality (including cases with unknown $M_f,L_f$ and $\phi_*$) without restarts or multi-threading, advancing practical, robust optimization for hybrid convex problems.

Abstract

This paper develops two parameter-free methods for solving convex and strongly convex hybrid composite optimization problems, namely, a composite subgradient type method and a proximal bundle type method. Functional complexity bounds for the two methods are established in terms of the unknown strong convexity parameter. The two proposed methods are universal with respect to all problem parameters, including the strong convexity one, and require no knowledge of the optimal value. Moreover, in contrast to previous works, they do not restart nor use multiple threads.

Theorems & Definitions (33)

• Lemma 2.1
• proof
• Theorem 2.2
• Theorem 2.3
• Proposition 3.1
• Theorem 3.2
• proof
• Lemma 3.3
• proof
• Lemma 3.4