Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Computational Lower Bounds for Regret Minimization in Normal-Form Games

Ioannis Anagnostides, Alkis Kalavasis, Tuomas Sandholm

TL;DR

Evidence is provided that existing learning algorithms, such as multiplicative weights update, are close to optimal and lower bounds for the problem of computing a CE that can be expressed as a uniform mixture of product distributions are proved.

Abstract

A celebrated connection in the interface of online learning and game theory establishes that players minimizing swap regret converge to correlated equilibria (CE) -- a seminal game-theoretic solution concept. Despite the long history of this problem and the renewed interest it has received in recent years, a basic question remains open: how many iterations are needed to approximate an equilibrium under the usual normal-form representation? In this paper, we provide evidence that existing learning algorithms, such as multiplicative weights update, are close to optimal. In particular, we prove lower bounds for the problem of computing a CE that can be expressed as a uniform mixture of $T$ product distributions -- namely, a uniform $T$-sparse CE; such lower bounds immediately circumscribe (computationally bounded) regret minimization algorithms in games. Our results are obtained in the algorithmic framework put forward by Kothari and Mehta (STOC 2018) in the context of computing Nash equilibria, which consists of the sum-of-squares (SoS) relaxation in conjunction with oracle access to a verification oracle; the goal in that framework is to lower bound either the degree of the SoS relaxation or the number of queries to the verification oracle. Here, we obtain two such hardness results, precluding computing i) uniform $\text{log }n$-sparse CE when $ε=\text{poly}(1/\text{log }n)$ and ii) uniform $n^{1 - o(1)}$-sparse CE when $ε= \text{poly}(1/n)$.

Computational Lower Bounds for Regret Minimization in Normal-Form Games

TL;DR

Evidence is provided that existing learning algorithms, such as multiplicative weights update, are close to optimal and lower bounds for the problem of computing a CE that can be expressed as a uniform mixture of product distributions are proved.

Abstract

A celebrated connection in the interface of online learning and game theory establishes that players minimizing swap regret converge to correlated equilibria (CE) -- a seminal game-theoretic solution concept. Despite the long history of this problem and the renewed interest it has received in recent years, a basic question remains open: how many iterations are needed to approximate an equilibrium under the usual normal-form representation? In this paper, we provide evidence that existing learning algorithms, such as multiplicative weights update, are close to optimal. In particular, we prove lower bounds for the problem of computing a CE that can be expressed as a uniform mixture of product distributions -- namely, a uniform -sparse CE; such lower bounds immediately circumscribe (computationally bounded) regret minimization algorithms in games. Our results are obtained in the algorithmic framework put forward by Kothari and Mehta (STOC 2018) in the context of computing Nash equilibria, which consists of the sum-of-squares (SoS) relaxation in conjunction with oracle access to a verification oracle; the goal in that framework is to lower bound either the degree of the SoS relaxation or the number of queries to the verification oracle. Here, we obtain two such hardness results, precluding computing i) uniform -sparse CE when and ii) uniform -sparse CE when .

Paper Structure

This paper contains 29 sections, 27 theorems, 69 equations, 2 figures.

Table of Contents

  1. Introduction
  2. The algorithmic framework
  3. Preliminaries
  4. Two-player games
  5. Correlated equilibria and sparsity
  6. Social welfare
  7. SoS and pseudo-equilibria
  8. Oblivious rounding with a verification oracle
  9. Our results
  10. Technical overview
  11. SoS hardness for the welfare maximization variant
  12. Hardness for enumeration algorithms
  13. Combining the games
  14. Related work
  15. Further preliminaries
  16. ...and 14 more sections

Key Result

Theorem 1.7

Suppose that there is a degree-$d$, $q$-query $\textsf{OV}$ rounding algorithm for uniform $n^{1 - o(1)}$-sparse $\epsilon$-CE, with $\epsilon = n^{-c}$ for some constant $c$. Then, either $d = 2^{ \Omega( \sqrt{ \log n \log \log n } )}$ or $q = 2^{\Omega(n)}$.

Figures (2)

  • Figure 1: Different correlated distributions when $k = 4$ (up to the normalization constant $32$).
  • Figure 2: An example of $\mathbf{R}^S$ when $n = 6$, $\ell = 4$, and $S = \{1, 2, 3, 4\}$. Entries highlighted in gray correspond to strictly dominated actions for Player $x$.

Theorems & Definitions (55)

  • Definition 1.1
  • Definition 1.2: Sparse distribution
  • Definition 1.2: Pseudo-distribution
  • Definition 1.3
  • Definition 1.4: Oblivious rounding algorithm
  • Definition 1.5: Verification oracle
  • Definition 1.6
  • Theorem 1.7
  • Theorem 1.8
  • Theorem 1.9
  • ...and 45 more