Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Equilibria under Dynamic Benchmark Consistency in Non-Stationary Multi-Agent Systems

Ludovico Crippa, Yonatan Gur, Bar Light

TL;DR

In a broad range of multi-agent systems where non-stationarity is prevalent, algorithms designed to compete with dynamic benchmarks can improve both individual and welfare guarantees, and their emerging dynamics approximate a sequence of static equilibrium outcomes.

Abstract

We formulate and study a general time-varying multi-agent system where players repeatedly compete under incomplete information. Our work is motivated by scenarios commonly observed in online advertising and retail marketplaces, where agents and platform designers optimize algorithmic decision-making in dynamic competitive settings. In these systems, no-regret algorithms that provide guarantees relative to \emph{static} benchmarks can perform poorly and the distributions of play that emerge from their interaction do not correspond anymore to static solution concepts such as coarse correlated equilibria. Instead, we analyze the interaction of \textit{dynamic benchmark} consistent policies that have performance guarantees relative to \emph{dynamic} sequences of actions, and through a novel \textit{tracking error} notion we delineate when their empirical joint distribution of play can approximate an evolving sequence of static equilibria. In systems that change sufficiently slowly (sub-linearly in the horizon length), we show that the resulting distributions of play approximate the sequence of coarse correlated equilibria, and apply this result to establish improved welfare bounds for smooth games. On a similar vein, we formulate internal dynamic benchmark consistent policies and establish that they approximate sequences of correlated equilibria. Our findings therefore suggest that in a broad range of multi-agent systems where non-stationarity is prevalent, algorithms designed to compete with dynamic benchmarks can improve both individual and welfare guarantees, and their emerging dynamics approximate a sequence of static equilibrium outcomes.

Equilibria under Dynamic Benchmark Consistency in Non-Stationary Multi-Agent Systems

TL;DR

In a broad range of multi-agent systems where non-stationarity is prevalent, algorithms designed to compete with dynamic benchmarks can improve both individual and welfare guarantees, and their emerging dynamics approximate a sequence of static equilibrium outcomes.

Abstract

We formulate and study a general time-varying multi-agent system where players repeatedly compete under incomplete information. Our work is motivated by scenarios commonly observed in online advertising and retail marketplaces, where agents and platform designers optimize algorithmic decision-making in dynamic competitive settings. In these systems, no-regret algorithms that provide guarantees relative to \emph{static} benchmarks can perform poorly and the distributions of play that emerge from their interaction do not correspond anymore to static solution concepts such as coarse correlated equilibria. Instead, we analyze the interaction of \textit{dynamic benchmark} consistent policies that have performance guarantees relative to \emph{dynamic} sequences of actions, and through a novel \textit{tracking error} notion we delineate when their empirical joint distribution of play can approximate an evolving sequence of static equilibria. In systems that change sufficiently slowly (sub-linearly in the horizon length), we show that the resulting distributions of play approximate the sequence of coarse correlated equilibria, and apply this result to establish improved welfare bounds for smooth games. On a similar vein, we formulate internal dynamic benchmark consistent policies and establish that they approximate sequences of correlated equilibria. Our findings therefore suggest that in a broad range of multi-agent systems where non-stationarity is prevalent, algorithms designed to compete with dynamic benchmarks can improve both individual and welfare guarantees, and their emerging dynamics approximate a sequence of static equilibrium outcomes.
Paper Structure (41 sections, 17 theorems, 154 equations, 5 figures, 2 tables, 3 algorithms)

This paper contains 41 sections, 17 theorems, 154 equations, 5 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Motivation
  3. Limitation of static benchmarks in non-stationary multi-agent systems
  4. Main contributions
  5. Literature Review
  6. Model
  7. Primitives and notations
  8. Dynamic benchmark consistency
  9. Equilibria and tracking error
  10. Discussion and extensions
  11. Random payoffs.
  12. Piece-wise stationary measures.
  13. Characterizing joint distributions of play and their tracking error
  14. Repeated games
  15. Time-varying games
  16. ...and 26 more sections

Key Result

Proposition 1

Let $C$ and $\tilde{C}$ be two non-decreasing sub-linear sequences, and $\tilde{\pi} \in \mathcal{P}_{DB}^N(\tilde{C})$ be a profile of DB($\tilde{C}$) consistent policies. Take any convergent sub-sequence of the joint distribution of play $\bar{\delta}_T^{\tilde{\pi}}$ and denote its limit by $\ti

Figures (5)

  • Figure 1: Distance between the average empirical joint distribution of play and the coarse correlated equilibrium of each stage game over the first and second halves of the season in Example \ref{['pricing game example']}. Sellers employ the Exp3 algorithm tuned with exploration parameter $\gamma_T =~\sqrt{2 \ln(2)/((e-1)T)}$. The figure shows the $\ell_2$-distance of the empirical joint distribution of play in the first and second half from $(p_h,p_h)$ and $(p_l, p_l)$ respectively. The empirical joint distribution of play is estimated by averaging the results over $750$ simulations.
  • Figure 2: Distance between the empirical joint distribution of play and the coarse correlated equilibrium of each stage game over the first and second halves of the season in Example \ref{['pricing game example']} together with resulting tracking error. Sellers employ the Exp3S auer2002nonstochastic algorithm tuned with exploration parameter $\gamma_T = \sqrt{4 \ln(T)/ T}$ and sharing factor $\alpha_T = 2/T$. The figure shows the $\ell_2$-distance of the distribution of play in the first and second half from $(p_h,p_h)$ and $(p_l, p_l)$ respectively. The distribution of play is estimated by averaging the results over $750$ simulations.
  • Figure : (a) Running mean prices for the two sellers (restricted to each time batch where the game is constant) when $T =~10^5$.
  • Figure : (a) Running mean prices for the two sellers (restricted to each time batch where the game is constant) when $T =~10^5$.
  • Figure : (b) Tracking error as a function of the horizon length $T$.

Theorems & Definitions (42)

  • Example 1: Evolving market conditions
  • Definition 1: Dynamic benchmark consistency
  • Definition 2: Tracking error
  • Proposition 1: Approximation of joint distribution of play
  • Theorem 1: Joint distributions of play under dynamic benchmark consistency
  • Theorem 2: Sufficient conditions for a diminishing tracking error
  • Theorem 3: Necessary conditions for a diminishing tracking error
  • Theorem 4: Tracking the sequence of coarse correlated equilibria
  • Definition 3: Price of anarchy
  • Corollary 1: PoA equivalence
  • ...and 32 more