Lower Bounds for Frank-Wolfe on Strongly Convex Sets

TL;DR

This work investigates whether strong convexity of the constraint set can yield faster convergence for the Frank–Wolfe algorithm beyond the known $\mathcal{O}(1/\sqrt{\varepsilon})$ rate. Focusing on the model problem of minimizing $f(x)=\|x-p\|_2^2$ over the unit ball with $\|p\|=1$, the authors exploit a two-dimensional invariant subspace to analyze FW dynamics via forward-backward trajectory construction. They develop a specialized backward reconstruction technique to synthesize long, high-precision worst-case trajectories and prove a $\Omega(1/\sqrt{\varepsilon})$ lower bound for exact line search and short steps, extending to ellipsoids by affine invariance. The results demonstrate that optimizer position materially affects convergence and that set smoothness alone does not enable a faster uniform rate, highlighting fundamental limits for projection-free methods in this geometric setting. These findings motivate exploring algorithmic variants tailored to strongly convex geometry that could potentially beat the bound while maintaining projection-free advantages.

Abstract

We present a constructive lower bound of $Ω(1/\sqrt{\varepsilon})$ for Frank-Wolfe (FW) when both the objective and the constraint set are smooth and strongly convex, showing that the known uniform $\mathcal{O}(1/\sqrt{\varepsilon})$ guarantees in this regime are tight. It is known that under additional assumptions on the position of the optimizer, FW can converge linearly. However, it remained unclear whether strong convexity of the set can yield rates uniformly faster than $\mathcal{O}(1/\sqrt{\varepsilon})$, i.e., irrespective of the position of the optimizer. To investigate this question, we focus on a simple yet representative problem class: minimizing a strongly convex quadratic over the Euclidean unit ball, with the optimizer on the boundary. We analyze the dynamics of FW for this problem in detail and develop a novel computational approach to construct worst-case FW trajectories, which is of independent interest. Guided by these constructions, we develop an analytical proof establishing the lower bound.

Figures (8)

• Figure 1: Log–log plot of the error versus iteration for with exact line search on the function $f(x) = \Vert x - p\Vert_2^2$ over the Euclidean unit ball for different positions of the optimizer $p$ with multiple initializations.
• Figure 2: Convergence landscape of for \ref{['eq:Pball']} with target at $p=(0,1)$ denoted by black marker. Colors indicate the number of iterations to reach $10^{-4}$ accuracy from each point in the Euclidean unit ball.
• Figure 3: Log-log plot of the error versus iteration for fw with exact line search on the Euclidean unit ball, shown for multiple initializations.
• Figure 4: Phase diagram of the forward dynamics in the $(r, s)$ state space. The red region indicates the stable phase where the contraction factor increases ($s_{t+1} > s_t$), while the blue region highlights the unstable regime where the contraction improves ($s_{t+1} < s_t$) and jumps occur. The boundary curve $s = g(r)$ separates these two regimes.
• Figure 5: Grid search for problem \ref{['eq:P_max_stable_phase']}, showing the length of the stable phase $\tau$ as a function of $s_0$ for $r_0=1$

Theorems & Definitions (36)

• lemma 0: Invariant Subspace
• proposition 0: Two-step termination
• proposition 0
• proposition 0
• proposition 0: Jump characterization
• theorem 1: Lower Bound
• lemma 1: Invariant Subspace
• proof
• proposition 1: Two-step termination
• proof