Convergence of linesearch-based generalized conditional gradient methods without smoothness assumptions
Shotaro Yagishita
TL;DR
This paper addresses minimizing $F(x)=f(x)+g(x)$ over a composite objective without smoothness assumptions on $f$. It analyzes two linesearch-based generalized conditional gradient methods: a nonmonotone average-type Armijo variant and a parameter-free variant that adapts to Hölder regularity of $\nabla f$. The main results establish subsequential convergence: the generated sequences are nonincreasing in $F$, accumulate to stationary points ($G(x^*)=0$), and under mild level-set boundedness, $\min_{0\le l\le k} G(x^l)\to0$; the parameter-free variant achieves these properties without requiring Hölder continuity and automatically adapts if it holds. These findings extend convergence analyses to settings without smoothness and support practical use of linesearch-based GCGMs in rough-gradient regimes, while pointing to future work on conditions ensuring full-sequence convergence.
Abstract
The generalized conditional gradient method is a popular algorithm for solving composite problems whose objective function is the sum of a smooth function and a nonsmooth convex function. Many convergence analyses of the algorithm rely on smoothness assumptions, such as the Lipschitz continuity of the gradient of the smooth part. This paper provides convergence results of linesearch-based generalized conditional gradient methods without smoothness assumptions. In particular, we show that a parameter-free variant, which automatically adapts to the Hölder exponent, guarantees convergence even when the gradient of the smooth part of the objective is not Hölder continuous.