Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Barriers to Welfare Maximization with No-Regret Learning

Ioannis Anagnostides, Alkis Kalavasis, Tuomas Sandholm

TL;DR

This paper establishes the first computational lower bounds for that problem in two-player (general-sum) games under the constraint that the CCE reached approximates the optimal social welfare (or some other natural objective) under the constraint that the CCE reached approximates the optimal social welfare.

Abstract

A celebrated result in the interface of online learning and game theory guarantees that the repeated interaction of no-regret players leads to a coarse correlated equilibrium (CCE) -- a natural game-theoretic solution concept. Despite the rich history of this foundational problem and the tremendous interest it has received in recent years, a basic question still remains open: how many iterations are needed for no-regret players to approximate an equilibrium? In this paper, we establish the first computational lower bounds for that problem in two-player (general-sum) games under the constraint that the CCE reached approximates the optimal social welfare (or some other natural objective). From a technical standpoint, our approach revolves around proving lower bounds for computing a near-optimal $T$-sparse CCE -- a mixture of $T$ product distributions, thereby circumscribing the iteration complexity of no-regret learning even in the centralized model of computation. Our proof proceeds by extending a classical reduction of Gilboa and Zemel [1989] for optimal Nash to sparse (approximate) CCE. In particular, we show that the inapproximability of maximum clique precludes attaining any non-trivial sparsity in polynomial time. Moreover, we strengthen our hardness results to apply in the low-precision regime as well via the planted clique conjecture.

Barriers to Welfare Maximization with No-Regret Learning

TL;DR

This paper establishes the first computational lower bounds for that problem in two-player (general-sum) games under the constraint that the CCE reached approximates the optimal social welfare (or some other natural objective) under the constraint that the CCE reached approximates the optimal social welfare.

Abstract

A celebrated result in the interface of online learning and game theory guarantees that the repeated interaction of no-regret players leads to a coarse correlated equilibrium (CCE) -- a natural game-theoretic solution concept. Despite the rich history of this foundational problem and the tremendous interest it has received in recent years, a basic question still remains open: how many iterations are needed for no-regret players to approximate an equilibrium? In this paper, we establish the first computational lower bounds for that problem in two-player (general-sum) games under the constraint that the CCE reached approximates the optimal social welfare (or some other natural objective). From a technical standpoint, our approach revolves around proving lower bounds for computing a near-optimal -sparse CCE -- a mixture of product distributions, thereby circumscribing the iteration complexity of no-regret learning even in the centralized model of computation. Our proof proceeds by extending a classical reduction of Gilboa and Zemel [1989] for optimal Nash to sparse (approximate) CCE. In particular, we show that the inapproximability of maximum clique precludes attaining any non-trivial sparsity in polynomial time. Moreover, we strengthen our hardness results to apply in the low-precision regime as well via the planted clique conjecture.

Paper Structure

This paper contains 23 sections, 26 theorems, 50 equations, 3 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Our results
  3. Technical approach
  4. Low-precision regime
  5. Further related work
  6. Preliminaries
  7. Notation
  8. Two-player games and equilibrium concepts
  9. Two-player games
  10. Sparse CCE
  11. Sparse equilibria and regret
  12. Social welfare
  13. Maximum clique and planted clique
  14. Planted clique problem
  15. The complexity of computing near-optimal sparse CCE
  16. ...and 8 more sections

Key Result

Theorem 1.1

$\textsc{OptimalSparseCCE}({\mathcal{G}}, n^{1 - \epsilon}, n^{-c}, n^{-c})$ with respect to $n \times n$ games is $\NP$-hard for any constant $\epsilon > 0$ and some constant $c$.

Figures (3)

  • Figure 1: A $4$-node graph $G$.
  • Figure 2: The payoff matrix $2 \cdot \mathbf{R} = 2 \cdot \mathbf{R}(G)$ per \ref{['eq:game']}.
  • Figure 3: Payoff matrices of ${\mathcal{G}}'$ (after multiplying each entry by $2$) based on graph $G$ of \ref{['fig:graph']}.

Theorems & Definitions (41)

  • Theorem 1.1
  • Theorem 1.2
  • Definition 2.1: Sparse (correlated) distributions
  • Definition 2.2: Coarse correlated equilibrium
  • Proposition 2.3
  • Theorem 2.4: Zuckerman07:Linear, Zuckerman07:Linear
  • Conjecture 2.5
  • Theorem 3.1
  • Corollary 3.2
  • Theorem 3.2
  • ...and 31 more