Robustness of the Frank-Wolfe Method under Inexact Oracles and the Cost of Linear Minimization
Tao Hu
TL;DR
This work analyzes the robustness of the projection-free Frank-Wolfe method under inexact gradient information modeled by a $\delta$-oracle, and it demonstrates that oracle errors do not accumulate asymptotically. For $L$-smooth nonconvex objectives, the authors prove a convergence bound of $\min_{0\le k\le K} G(x^k) \le \sqrt{\frac{2C\,(f(x^0)-f^*)}{K+1}}+2\delta$, with a stronger, directionally relative $\delta$-oracle yielding residual terms that scale with $\|\nabla f(x^k)\|$, and a potential residual-free rate under an interior margin. They further relate projection error to LMO accuracy, showing that a $K$-approximate projection at a scaled point yields an $\varepsilon$-accurate LMO with $\varepsilon=O((K+\frac{1}{2}\delta_C^2+\mu_C\delta_C)/\lambda)$, implying approximate projections cannot be universally cheaper than accurate LMOs. These results substantiate the FW method’s robustness and clarify the cost dynamics between projection and LMO subproblems in constrained optimization.
Abstract
We investigate the robustness of the Frank-Wolfe method when gradients are computed inexactly and examine the relative computational cost of the linear minimization oracle (LMO) versus projection. For smooth nonconvex functions, we establish a convergence guarantee of order $\mathcal{O}(1/\sqrt{k}+δ)$ for Frank-Wolfe with a $δ$--oracle. Our results strengthen previous analyses for convex objectives and show that the oracle errors do not accumulate asymptotically. We further prove that approximate projections cannot be computationally cheaper than accurate LMOs, thus extending to the case of inexact projections. These findings reinforce the robustness and efficiency of the Frank-Wolfe framework.