Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sink equilibria and the attractors of learning in games

Oliver Biggar, Christos Papadimitriou

TL;DR

This work investigates the long-run outcomes of learning dynamics, focusing on the replicator dynamic and its suspected link to sink equilibria from the preference graph. It disproves the one-to-one conjecture by constructing counterexamples that hinge on local sources, and it introduces pseudoconvexity as a sufficient condition ensuring that the content of a two-player sink equilibrium is an attractor. A Lyapunov-type analysis in correlated space using a product matrix provides the stability result, while the results delineate both the limits of sink-equilibria-based predictions and the new avenues needed to characterize attractors in broader settings. The findings sharpen our understanding of learning in games and point to algorithmic and complexity-related questions for scaling to larger games and other dynamics.

Abstract

Characterizing the limit behavior -- that is, the attractors -- of learning dynamics is one of the most fundamental open questions in game theory. In recent work on this front, it was conjectured that the attractors of the replicator dynamic are in one-to-one correspondence with the sink equilibria of the game -- the sink strongly connected components of a game's preference graph -- , and it was established that they do stand in at least one-to-many correspondence with them. Here, we show that the one-to-one conjecture is false. We disprove this conjecture over the course of three theorems: the first disproves a stronger form of the conjecture, while the weaker form is disproved separately in the two-player and $N$-player ($N>2$) cases. By showing how the conjecture fails, we lay out the obstacles that lie ahead for characterizing attractors of the replicator, and introduce new ideas with which to tackle them. All three counterexamples derive from an object called a local source -- a point lying within the sink equilibrium, and yet which is `locally repelling'; we prove that the absence of local sources is necessary, but not sufficient, for the one-to-one property to be true. We complement this with a sufficient condition: we introduce a local property of a sink equilibrium called pseudoconvexity, and establish that when the sink equilibria of a two-player game are pseudoconvex then they precisely define the attractors. Pseudoconvexity generalizes the previous cases -- such as zero-sum games and potential games -- where this conjecture was known to hold, and reformulates these cases in terms of a simple graph property.

Sink equilibria and the attractors of learning in games

TL;DR

This work investigates the long-run outcomes of learning dynamics, focusing on the replicator dynamic and its suspected link to sink equilibria from the preference graph. It disproves the one-to-one conjecture by constructing counterexamples that hinge on local sources, and it introduces pseudoconvexity as a sufficient condition ensuring that the content of a two-player sink equilibrium is an attractor. A Lyapunov-type analysis in correlated space using a product matrix provides the stability result, while the results delineate both the limits of sink-equilibria-based predictions and the new avenues needed to characterize attractors in broader settings. The findings sharpen our understanding of learning in games and point to algorithmic and complexity-related questions for scaling to larger games and other dynamics.

Abstract

Characterizing the limit behavior -- that is, the attractors -- of learning dynamics is one of the most fundamental open questions in game theory. In recent work on this front, it was conjectured that the attractors of the replicator dynamic are in one-to-one correspondence with the sink equilibria of the game -- the sink strongly connected components of a game's preference graph -- , and it was established that they do stand in at least one-to-many correspondence with them. Here, we show that the one-to-one conjecture is false. We disprove this conjecture over the course of three theorems: the first disproves a stronger form of the conjecture, while the weaker form is disproved separately in the two-player and -player () cases. By showing how the conjecture fails, we lay out the obstacles that lie ahead for characterizing attractors of the replicator, and introduce new ideas with which to tackle them. All three counterexamples derive from an object called a local source -- a point lying within the sink equilibrium, and yet which is `locally repelling'; we prove that the absence of local sources is necessary, but not sufficient, for the one-to-one property to be true. We complement this with a sufficient condition: we introduce a local property of a sink equilibrium called pseudoconvexity, and establish that when the sink equilibria of a two-player game are pseudoconvex then they precisely define the attractors. Pseudoconvexity generalizes the previous cases -- such as zero-sum games and potential games -- where this conjecture was known to hold, and reformulates these cases in terms of a simple graph property.

Paper Structure

This paper contains 14 sections, 12 theorems, 15 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Refuting the Conjectures
  3. Local sources and Conjecture \ref{['conj: strong content conjecture']}
  4. Games with (at least) three players
  5. Games with two players
  6. Pseudoconvex sink equilibria are attractors
  7. Understanding pseudoconvexity
  8. Stability of pseudoconvex sink equilibria
  9. Conclusions and open problems
  10. Local sources and paths.
  11. Beyond uniform convergence.
  12. Algorithmic problems.
  13. Large games.
  14. Omitted proofs

Key Result

Lemma 2.2

If $H$ is a sink equilibrium, and $H$ has a local source, then $\mathop{\mathrm{content}}\nolimits(H)$ is not an attractor.

Figures (7)

  • Figure 1: The preference graph of Shapley's gameshapley_topics_1964, and a typical payoff matrix representation. It has a unique sink equilibrium, which is the highlighted 6-cycle.
  • Figure 2: A preference graph (Fig. \ref{['fig:cog']}) whose sink equilibrium (highlighted in gray) has a local source $a$ (Def. \ref{['def: local source']}). The point $\hat{x}$ represents the interior fixed point of the $2\times 2$ subgame in the top left, shown separately in Fig. \ref{['fig:cog local source']}. The presence of the local source at $a$ implies that any replicator attractor of the game which contains $a$ must also contain $\hat{x}$, disproving Conjecture \ref{['conj: strong content conjecture']}.
  • Figure 3: A 3-player counterexample. We show that a replicator trajectory exists from $a$ to $\hat{x}$ (a fully-mixed Nash equilibrium of the subgame in \ref{['fig:small_3player']}) and also from $\hat{x}$ to $b$, implying that any attractor containing $a$ must also contain $b$.
  • Figure 4: A two-player counterexample. We show that replicator trajectories exist from $a$ to $\hat{x}$ to $\hat{y}$ to $c$ to $b$, meaning that $b$ must be in any attractor containing $a$.
  • Figure 5: A cavity (Def. \ref{['def:corner']}) of a sink equilibrium $H$, where $x\not\in H$ and $w,y,z\in H$. Because $x\not\in H$, the arcs at $x$ are necessarily directed towards $y$ and $z$ respectively. Up to symmetry, there are three cases, shown in Figs. \ref{['fig:corner 1']}, \ref{['fig:corner 2']} and \ref{['fig:corner 3']}.
  • ...and 2 more figures

Theorems & Definitions (27)

  • Conjecture 1.1
  • Conjecture 1.2
  • Definition 1.3
  • Definition 2.1: Local sources
  • Lemma 2.2
  • Lemma 2.3
  • proof
  • Lemma 2.4
  • Definition 3.1
  • Definition 3.2
  • ...and 17 more