Proximal gradient descent on the smoothed duality gap to solve saddle point problems
Olivier Fercoq
TL;DR
The paper tackles convex-concave saddle-point problems by recasting them as the minimization of a weakly convex merit function, the Self-Centered Smoothed Duality Gap $G_\beta(z) = F(z) + F^_{\beta,M}^*(z)$. It develops proximal-gradient methods, including accelerated and restarted variants, to minimize $G_\beta$ with a decreasing smoothing sequence $\beta_k$, achieving convergence to saddle points and competitive empirical performance. Theoretical results show worst-case complexity similar to PDHG-type methods and potential linear convergence in favorable regimes, while numerical experiments on small LPs and large SOC programs validate the approach and highlight practical gains from acceleration and restarts. The work opens avenues for integrating function-minimization techniques into primal-dual frameworks and suggests extensions to coordinate-descent, line search, and non-convex non-concave saddle-point problems.
Abstract
In this paper, we minimize the self-centered smoothed gap, a recently introduced optimality measure, in order to solve convex-concave saddle point problems. The self-centered smoothed gap can be computed as the sum of a convex, possibly nonsmooth function and a smooth weakly convex function. Although it is not convex, we propose an algorithm that minimizes this quantity, effectively reducing convex-concave saddle point problems to a minimization problem. Its worst case complexity is comparable to the one of the restarted and averaged primal dual hybrid gradient method, and the algorithm enjoys linear convergence in favorable cases.