Contractivity and linear convergence in bilinear saddle-point problems: An operator-theoretic approach
Colin Dirren, Mattia Bianchi, Panagiotis D. Grontas, John Lygeros, Florian Dörfler
TL;DR
This work analyzes the convex-concave bilinear saddle-point problem by casting it as a zero of a saddle-point operator and solving it via a preconditioned forward-backward scheme. It introduces an operator-theoretic framework, including the novel notion of inverse Lipschitz operators, to establish contractivity and linear convergence for three primal-dual algorithms (Chambolle–Pock, a semi-implicit method, and a fully explicit preconditioned method) under three key conditions C1–C3. The main contributions are tighter, contractivity-based convergence results, explicit step-size regimes, and new contractivity guarantees under C2 and C3, expanding the regime where linear convergence can be guaranteed. These results yield improved rates over prior work, provide robustness insights, and lay the groundwork for extensions to accelerated, stochastic, and time-varying settings in large-scale machine learning tasks.
Abstract
We study the convex-concave bilinear saddle-point problem $\min_x \max_y f(x) + y^\top Ax - g(y)$, where both, only one, or none of the functions $f$ and $g$ are strongly convex, and suitable rank conditions on the matrix $A$ hold. The solution of this problem is at the core of many machine learning tasks. By employing tools from monotone operator theory, we systematically prove the contractivity (in turn, the linear convergence) of several first-order primal-dual algorithms, including the Chambolle-Pock method. Our approach results in concise proofs, and it yields new convergence guarantees and tighter bounds compared to known results.