A Linearly Convergent Frank-Wolfe-type Method for Smooth Convex Minimization over the Spectrahedron
Dan Garber
TL;DR
This work tackles smooth convex minimization over the spectrahedron $\mathcal{S}^n$ by introducing a Frank-Wolfe-type method that relies solely on rank-one updates. Under quadratic growth and strict complementarity, the algorithm achieves linear convergence after a finite burn-in that is independent of the ambient dimension, while maintaining $O(n^2)$ time per iteration and requiring only the smoothness parameter $\beta$. The method combines standard FW steps with specialized away/drop and randomized pairwise updates to adapt the active face and ensure contraction, achieving linear rates for both rank-one and higher-rank optima. Empirical results on synthetic matrix sensing corroborate the theory, showing linear convergence where standard FW is sublinear, and highlighting robustness to violations of strict complementarity.
Abstract
We consider the problem of minimizing a smooth and convex function over the $n$-dimensional spectrahedron -- the set of real symmetric $n\times n$ positive semidefinite matrices with unit trace, which underlies numerous applications in statistics, machine learning and additional domains. Standard first-order methods often require high-rank matrix computations which are prohibitive when the dimension $n$ is large. The well-known Frank-Wolfe method on the other hand, only requires efficient rank-one matrix computations, however suffers from worst-case slow convergence, even under conditions that enable linear convergence rates for standard methods. In this work we present the first Frank-Wolfe-based algorithm that only applies efficient rank-one matrix computations and, assuming quadratic growth and strict complementarity conditions, is guaranteed, after a finite number of iterations, to converges linearly, in expectation, and independently of the ambient dimension.