A potentialization algorithm for games with applications to multi-agent learning in repeated games

TL;DR

An algorithm that assigns to any game in normal form an approximating game that admits an ordinal potential function guarantees convergence to stable agent behavior.

Abstract

We investigate an algorithm that assigns to any game in normal form an approximating game that admits an ordinal potential function. Due to the properties of potential games, the algorithm equips every game with a surrogate reward structure that allows efficient multi-agent learning. Numerical simulations using the replicator dynamics show that 'potentialization' guarantees convergence to stable agent behavior.

Figures (5)

• Figure 1: Construction of the potential function.
• Figure 2: Coordination game. The pure Nash equilibria of the original game are Pareto-ordered: $\alpha > \beta > \gamma$. The second and the third action of both players is weakly dominated. There is a cycle containing all vertices but $\alpha$. All pure equilibria are retained and two additional pure equilibria are created.
• Figure 3: Game from Voorneveld and Nolde (1997). The vertices denoted by $\bullet$ form a cycle. All three equilibria are retained but the Pareto-best original equilibrium becomes preferable to both players.
• Figure 4: Experimental results for $(10 \times 10)$-games ($\pm$ empirical standard deviation).
• Figure 5: Experimental results for $(4 \times 4 \times 4)$-games ($\pm$ empirical standard deviation).

Theorems & Definitions (10)

• Definition 1: Finite game in normal form
• Definition 2: Deviation graph
• Definition 3: Ordinal potential monderer1996potential
• Definition 4: Weak improvement cycle
• Theorem 5: Voorneveld and Nolde voorneveld1997
• Definition 6: Strongly connected component
• Definition 7: Condensation of weighted directed graphs
• Proposition 8
• proof
• Definition 9: Replicator differential equation taylor1978