Subgradient Gliding Method for Nonsmooth Convex Optimization

TL;DR

Numerical experiments demonstrate that the proposed novel subgradient gliding method converges reliably with a $100\% success rate and achieves orders-of-magnitude improvements in accuracy and convergence speed, which substantially expand the scope of subgradient-based optimization methods to non-Lipschitz nonsmooth convex problems.

Abstract

We identify and analyze a fundamental limitation of the classical projected subgradient method in nonsmooth convex optimization: the inevitable failure caused by the absence of valid subgradients at boundary points. We show that, under standard step sizes for both convex and strongly convex objectives, the method can fail after a single iteration with probability arbitrarily close to one, even on simple problem instances. To overcome this limitation, we propose a novel alternative termed the \textit{subgradient gliding method}, which remains well defined without boundary subgradients and avoids premature termination. Beyond resolving this foundational issue, the proposed framework encompasses the classical projected subgradient method as a special case and substantially enlarges its admissible step-size design space, providing greater flexibility for algorithmic design. We establish optimal ergodic convergence rates, $\mathcal{O}(1/\sqrt{t})$ for convex problems and $\mathcal{O}(1/t)$ for strongly convex problems, and further extend the framework to stochastic settings. Notably, our analysis does not rely on global Lipschitz continuity of the objective function, requiring only mild control on subgradient growth. Numerical experiments demonstrate that, in scenarios where the classical projected subgradient method fails completely, the proposed method converges reliably with a $100\%$ success rate and achieves orders-of-magnitude improvements in accuracy and convergence speed. These results substantially expand the scope of subgradient-based optimization methods to non-Lipschitz nonsmooth convex problems.

Figures (9)

• Figure 1: Lower bound on the area of intial points that lead to one-step termination of PSG.
• Figure 2: Iteration scheme of subgradient gliding method.
• Figure 3: Visualization of initial points where the classical PSG iteration succeeds or fails under different objective function settings.
• Figure 4: The impact of different choices of the gliding step size $\beta_s\equiv \beta$, where $\beta=0.1, 0.5, 0.9$ in the subgradient gliding method.
• Figure 5: Iteration comparison between classic PSG and the subgradient gliding method (SGM).

Theorems & Definitions (39)

• Theorem 1.1
• Example 1
• Example 2
• Example 3
• Theorem 2.1: One-step failure of the classical PSG in convex setting
• proof
• Theorem 2.2: One-step failure of the classical PSG in strongly convex setting
• proof
• Theorem 3.1
• proof