Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Simultaneous incremental support adjustment and metagame solving: An equilibrium-finding framework for continuous-action games

Carlos Martin, Tuomas Sandholm

TL;DR

This work presents a framework for computing approximate mixed-strategy Nash equilibria of continuous-action games, a modification of the traditional double oracle algorithm extended to multiple players and continuous action spaces that incrementally reduces the exploitability of the strategy profile in the finite metagame, pushing it toward Nash equilibrium.

Abstract

We present a framework for computing approximate mixed-strategy Nash equilibria of continuous-action games. It is a modification of the traditional double oracle algorithm, extended to multiple players and continuous action spaces. Unlike prior methods, it maintains fixed-cardinality pure strategy sets for each player. Thus, unlike prior methods, only a constant amount of memory is necessary. Furthermore, it does not require exact metagame solving on each iteration, which can be computationally expensive for large metagames. Moreover, it does not require global best-response computation on each iteration, which can be computationally expensive or even intractable for high-dimensional action spaces and general games. Our method incrementally reduces the exploitability of the strategy profile in the finite metagame, pushing it toward Nash equilibrium. Simultaneously, it incrementally improves the pure strategies that best respond to this strategy profile in the full game. We evaluate our method on various continuous-action games, showing that it obtains approximate mixed-strategy Nash equilibria with low exploitability.

Simultaneous incremental support adjustment and metagame solving: An equilibrium-finding framework for continuous-action games

TL;DR

This work presents a framework for computing approximate mixed-strategy Nash equilibria of continuous-action games, a modification of the traditional double oracle algorithm extended to multiple players and continuous action spaces that incrementally reduces the exploitability of the strategy profile in the finite metagame, pushing it toward Nash equilibrium.

Abstract

We present a framework for computing approximate mixed-strategy Nash equilibria of continuous-action games. It is a modification of the traditional double oracle algorithm, extended to multiple players and continuous action spaces. Unlike prior methods, it maintains fixed-cardinality pure strategy sets for each player. Thus, unlike prior methods, only a constant amount of memory is necessary. Furthermore, it does not require exact metagame solving on each iteration, which can be computationally expensive for large metagames. Moreover, it does not require global best-response computation on each iteration, which can be computationally expensive or even intractable for high-dimensional action spaces and general games. Our method incrementally reduces the exploitability of the strategy profile in the finite metagame, pushing it toward Nash equilibrium. Simultaneously, it incrementally improves the pure strategies that best respond to this strategy profile in the full game. We evaluate our method on various continuous-action games, showing that it obtains approximate mixed-strategy Nash equilibria with low exploitability.
Paper Structure (25 sections, 7 theorems, 4 equations, 9 figures, 1 table, 2 algorithms)

This paper contains 25 sections, 7 theorems, 4 equations, 9 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Problem formulation
  3. Related prior work
  4. Double oracle
  5. PSRO
  6. Proposed method
  7. Experiments
  8. Interval game
  9. Circle game
  10. Glicksberg--Gross game
  11. Continuous Colonel Blotto game
  12. Security game
  13. All-pay auction
  14. Chopstick auction
  15. Experimental results
  16. ...and 10 more sections

Key Result

Theorem 1

A constant-sum regular game has convex exploitability.

Figures (9)

  • Figure 1: Exploitabilities of learned mixed strategies (part 1).
  • Figure 2: Exploitabilities of learned mixed strategies (part 2).
  • Figure 3: Learned strategies for the circle game. Each dot shows an action and its opacity shows its probability.
  • Figure 4: Learned strategies for the Glicksberg--Gross game. The exact equilibrium strategies are shown in black.
  • Figure 5: Exact equilibrium for the 2-player, 3-item continuous Colonel Blotto game. Darkness shows density.
  • ...and 4 more figures

Theorems & Definitions (12)

  • Definition 1
  • Theorem 1
  • proof
  • Definition 2
  • Theorem 2
  • proof
  • Corollary 1
  • Corollary 2
  • Theorem 3
  • proof
  • ...and 2 more