Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Perturbing Best Responses in Zero-Sum Games

Adam Dziwoki, Rostislav Horcik

TL;DR

This work investigates how perturbing the best-response computations affects two cornerstone BRO-based algorithms for approximating Nash equilibria in two-player zero-sum games. It introduces perturbed BRO variants of Fictitious Play (SFP) and Double Oracle (SDO), analyzes their convergence under Gumbel and uniform perturbations, and shows that SFP attains $\mathcal{O}\left(\frac{\log n}{\varepsilon^2}\right)$ expected iterations to reach an $\varepsilon$-NE, while SDO achieves $\mathcal{O}(\log n)$ in expectation on certain challenging examples. The paper also develops efficient perturbation schemes for structured games (POSGs, EFGs) that perturb only terminal rewards or transitions, using clustering to reduce overhead; these methods speed convergence in several stochastic-game and grid-path experiments. Overall, perturbations extend the practical reach of BRO-based NE algorithms, enabling faster convergence in large or structured zero-sum games, with open questions regarding the full generality of DO under perturbations. $

Abstract

This paper investigates the impact of perturbations on the best-response-based algorithms approximating Nash equilibria in zero-sum games, namely Double Oracle and Fictitious Play. More precisely, we assume that the oracle computing the best responses perturbs the utilities before selecting the best response. We show that using such an oracle reduces the number of iterations for both algorithms. For some cases, suitable perturbations ensure the expected number of iterations is logarithmic. Although the utility perturbation is computationally demanding as it requires iterating through all pure strategies, we demonstrate that one can efficiently perturb the utilities in games where pure strategies have further inner structure.

Perturbing Best Responses in Zero-Sum Games

TL;DR

This work investigates how perturbing the best-response computations affects two cornerstone BRO-based algorithms for approximating Nash equilibria in two-player zero-sum games. It introduces perturbed BRO variants of Fictitious Play (SFP) and Double Oracle (SDO), analyzes their convergence under Gumbel and uniform perturbations, and shows that SFP attains expected iterations to reach an -NE, while SDO achieves in expectation on certain challenging examples. The paper also develops efficient perturbation schemes for structured games (POSGs, EFGs) that perturb only terminal rewards or transitions, using clustering to reduce overhead; these methods speed convergence in several stochastic-game and grid-path experiments. Overall, perturbations extend the practical reach of BRO-based NE algorithms, enabling faster convergence in large or structured zero-sum games, with open questions regarding the full generality of DO under perturbations. $

Abstract

This paper investigates the impact of perturbations on the best-response-based algorithms approximating Nash equilibria in zero-sum games, namely Double Oracle and Fictitious Play. More precisely, we assume that the oracle computing the best responses perturbs the utilities before selecting the best response. We show that using such an oracle reduces the number of iterations for both algorithms. For some cases, suitable perturbations ensure the expected number of iterations is logarithmic. Although the utility perturbation is computationally demanding as it requires iterating through all pure strategies, we demonstrate that one can efficiently perturb the utilities in games where pure strategies have further inner structure.

Paper Structure

This paper contains 10 sections, 9 theorems, 19 equations, 6 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. Stochastic Fictitious Play
  5. Stochastic Double Oracle
  6. Efficient Perturbations
  7. Experiments
  8. Conclusions
  9. Acknowledgments
  10. Appendix

Key Result

Lemma 1

Let $\bm{x}\in\mathbb{R}^n$ and $i^*\in\{1,\ldots,n\}$. Let $\bm{z}\in\mathbb{R}^n$ be a random vector whose components $z_i$ be drawn i.i.d. from $G(0,\beta)$. Then the probability

Figures (6)

  • Figure 1: An example of the 4-bit stochastic game.
  • Figure 2: An example of the 4-bit POSG.
  • Figure 3: The clusters corresponding to the terminal states.
  • Figure 4: SFP iterations on random $[0,1]$-matrix games (left), $\bm{U}^\top$ (middle), and the $f$-finger Morra game (right) for $\varepsilon=0.1$.
  • Figure 5: SDO iterations on $\bm{L}$ and $\bm{U}^\top$ with perturbations from $\mathcal{U}(-1,1)$ (left), $f$-finger Morra game (middle), and Colonel Blotto (right) for $\varepsilon=0.1$.
  • ...and 1 more figures

Theorems & Definitions (16)

  • Lemma 1
  • Corollary 2
  • Theorem 3
  • proof
  • Example 1
  • Lemma 4
  • proof
  • Lemma 5
  • proof
  • Theorem 6
  • ...and 6 more