A geometric decomposition of finite games: Convergence vs. recurrence under exponential weights

TL;DR

The paper introduces a Shahshahani-geometry-based Helmholtz-like decomposition for finite games, splitting any game into a potential part and an incompressible (harmonic) part. It shows that incompressible/harmonic games are volume-preserving under the Shahshahani metric and possess a constant of motion, leading to Poincaré recurrence for the continuous-time exponential weights dynamics. Crucially, harmonic and incompressible games are shown to coincide, connecting a dynamic symmetry with Candogan et al.'s strategic decomposition; the results imply convergence in potential games but quasi-periodic, recurrent behavior in harmonic ones. This framework provides a nuanced spectrum for learning in games and suggests broad extensions to other no-regret dynamics (FTRL) and richer dynamical settings.

Abstract

In view of the complexity of the dynamics of learning in games, we seek to decompose a game into simpler components where the dynamics' long-run behavior is well understood. A natural starting point for this is Helmholtz's theorem, which decomposes a vector field into a potential and an incompressible component. However, the geometry of game dynamics - and, in particular, the dynamics of exponential / multiplicative weights (EW) schemes - is not compatible with the Euclidean underpinnings of Helmholtz's theorem. This leads us to consider a specific Riemannian framework based on the so-called Shahshahani metric, and introduce the class of incompressible games, for which we establish the following results: First, in addition to being volume-preserving, the continuous-time EW dynamics in incompressible games admit a constant of motion and are Poincaré recurrent - i.e., almost every trajectory of play comes arbitrarily close to its starting point infinitely often. Second, we establish a deep connection with a well-known decomposition of games into a potential and harmonic component (where the players' objectives are aligned and anti-aligned respectively): a game is incompressible if and only if it is harmonic, implying in turn that the EW dynamics lead to Poincaré recurrence in harmonic games.

Figures (6)

• Figure 1: Unit balls on the orthant and the simplex under the Shahshahani metric (left and right respectively). Notice how the Shahshahani metric distorts distances near the boundary and flattens the balls along the axis that they are closest to.
• Figure 2: The evolution of \ref{['eq:EW']} in three randomly generated harmonic games with random initial conditions (left: $2\times2$; middle: $3\times2$; right: $2\times2\times2$). In all cases, the dynamics are Poincaré recurrent and cycle the game's NE (depicted in red). None of these games is zero-sum; the left and right games are strategically equivalent to a ZSG, while the middle is not – cf. \ref{['rem:harmonic-vs-zero-sum']}. For visual clarity, we have highlighted a randomly selected orbit in each case, and the arrows indicate the direction in which orbits are traversed. In \ref{['app:harmonic-potential-trajectories']} we include a series of trajectories of \ref{['eq:EW']}/\ref{['eq:RD']} for a convex combination of potential and a harmonic game, showing how Poincaré recurrence breaks down as the relative magnitude of the potential component increases.
• Figure 3: Maps \ref{['eq:simplex-parametrization']} and \ref{['eq:simplex-chart']} between the open corner of cube $\mathcal{C}^{\circ}$ in $\mathbb{R}^{m}$ and the open simplex $\mathcal{X}^{\circ}$ in $\mathbb{R}^{m+1}$ for $m = 2$. In light red are the tangent spaces $\mathop{\mathrm{T}}\nolimits\mathcal{C}^{\circ} = \mathbb{R}^{2}$ and $\mathop{\mathrm{T}}\nolimits\mathcal{X}^{\circ} = \mathcal{Z}$, where $\mathcal{Z}$ is the hyperplane $\mathcal{Z} = \setdef{(x_{0}, x_{1}, x_{2}) \in \mathbb{R}^{3}}{x_{0} + x_{1} + x_{2} = 0}$. The basis vectors $\tilde{e}_{1} = (1,0)$ and $\tilde{e}_{2} = (0,1)$ of $\mathop{\mathrm{T}}\nolimits\mathcal{C}^{\circ}$ are mapped by \ref{['eq:eff-rep-basis-vectors']} to the vectors $\tilde{e}_{1} = e_{1} - e_{0} = (-1, 1, 0)$ and $\tilde{e}_{2} = e_{2} - e_{0} = (-1, 0, 1)$ in $\mathcal{Z}$.
• Figure 4: A $(2 \times 2)$ game. Left: The strategy space of each player $i \in \{1,2\}$ in a $(2 \times 2)$ game is the $1$-dimensional open simplex $\mathcal{X}^{\circ}_{i}$ as a subspace of $\mathbb{R}^{2}$; the tangent space $\mathop{\mathrm{T}}\nolimits\mathcal{X}^{\circ}_{i}$ is the line $x_{0} + x_{1} = 0$. Right: The strategy space $\mathcal{X}^{\circ}_{1} \times\mathcal{X}^{\circ}_{2}$ of a $2 \times 2$ game is a subset of $\mathbb{R}^{4}$, so we represent the open corner of cube $\mathcal{C}^{\circ} = \mathcal{C}^{\circ}_{1} \times \mathcal{C}^{\circ}_{2} = \setdef{(\tilde{x}_{1}, \, \tilde{x}_{2})}{ \tilde{x}_{1} > 0, \, \tilde{x}_{2} > 0, \, \tilde{x}_{1} < 1, \, \tilde{x}_{2} < 1 }$ as an open subset of $\mathbb{R}^{2}$. Its tangent space $\mathop{\mathrm{T}}\nolimits\mathcal{C}^{\circ}$ is the whole $\mathbb{R}^{2}$.
• Figure 5: Euclidean (orange) vs. Shahshahani (blue) individual payoff gradients in a $2 \times 2$ potential and harmonic game (left and right respectively). Dark dotted lines represent payoff contours, and red dotted lines (left figure only) represent contours of the potential function. The replicator dynamical system \ref{['eq:logistic-system']} is equivalent to individual Shahshahani gradient ascent; the figure shows how the functional form of the inverse Shahshahani metric given by \ref{['eq:functional-inverse-sha-2-2']}, decaying to zero as the boundary is approached, is the key feature that confines RD (blue) to the interior of the strategy space, whereas Euclidean steepest individual payoff ascent (orange) leads to hitting the boundary in finite time. The payoffs used in these examples are $u_{1} =(2, 0, 3, 1), u_2 = (2, 3, 0, 1)$ in Prisoner's Dilemma, that is potential with potential function $\phi = (-1, 0, 0, 1)$; and $u_1 = (3, -3, -3, 3), u_2 = (-3, 3, 3, -3)$ in rescaled Matching Pennies, that is harmonic.

Theorems & Definitions (84)

• Proposition 1
• Definition 1
• Theorem 1
• Theorem 2
• Proposition 2
• Theorem 3
• Theorem 4
• Remark A.1
• Definition 2
• Lemma A.1: Components of gradient field