A Descent-based method on the Duality Gap for solving zero-sum games
Michail Fasoulakis, Evangelos Markakis, Giorgos Roussakis, Christodoulos Santorinaios
TL;DR
The paper addresses finding Nash equilibria in 2-player bilinear zero-sum games by exploiting the convex duality gap $V(\mathbf{x},\mathbf{y})$ and performing a descent on this function. It introduces a steepest-descent-like algorithm that moves in the direction minimizing the $\rho$-directional derivative of $V$, computed via small linear programs over $\rho$-best-response sets. The authors prove geometric convergence, derive iteration bounds of $O\left(\frac{1}{\rho \delta}\log\frac{1}{\delta}\right)$ to achieve a $\delta$-NE (and $O\left(\frac{1}{\sqrt{\delta}}\right)$ in a variant), and validate the approach experimentally against LP solvers and OGDA on large-scale games. The work demonstrates that descent-based methods on the duality gap can be competitive with standard approaches for zero-sum games and offers practical implementation strategies, including fixed-support LPs and adaptive step-sizing, to scale to thousands of strategies.
Abstract
We focus on the design of algorithms for finding equilibria in 2-player zero-sum games. Although it is well known that such problems can be solved by a single linear program, there has been a surge of interest in recent years for simpler algorithms, motivated in part by applications in machine learning. Our work proposes such a method, inspired by the observation that the duality gap (a standard metric for evaluating convergence in min-max optimization problems) is a convex function for bilinear zero-sum games. To this end, we analyze a descent-based approach, variants of which have also been used as a subroutine in a series of algorithms for approximating Nash equilibria in general non-zero-sum games. In particular, we study a steepest descent approach, by finding the direction that minimises the directional derivative of the duality gap function. Our main theoretical result is that the derived algorithms achieve a geometric decrease in the duality gap and improved complexity bounds until we reach an approximate equilibrium. Finally, we complement this with an experimental evaluation, which provides promising findings. Our algorithm is comparable with (and in some cases outperforms) some of the standard approaches for solving 0-sum games, such as OGDA (Optimistic Gradient Descent/Ascent), even with thousands of available strategies per player.