Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Equilibrium with Estimated Payoffs in Population Games

Shinkyu Park

TL;DR

This work studies a multi-agent decision problem in population games, where agents select from multiple available strategies and continually revise their selections based on the payoffs associated with these strategies, and proposes a design for a time-varies strategy revision rate to guarantee convergence.

Abstract

We study a multi-agent decision problem in population games, where agents select from multiple available strategies and continually revise their selections based on the payoffs associated with these strategies. Unlike conventional population game formulations, we consider a scenario where agents must estimate the payoffs through local measurements and communication with their neighbors. By employing task allocation games -- dynamic extensions of conventional population games -- we examine how errors in payoff estimation by individual agents affect the convergence of the strategy revision process. Our main contribution is an analysis of how estimation errors impact the convergence of the agents' strategy profile to equilibrium. Based on the analytical results, we propose a design for a time-varying strategy revision rate to guarantee convergence. Simulation studies illustrate how the proposed method for updating the revision rate facilitates convergence to equilibrium.

Learning Equilibrium with Estimated Payoffs in Population Games

TL;DR

This work studies a multi-agent decision problem in population games, where agents select from multiple available strategies and continually revise their selections based on the payoffs associated with these strategies, and proposes a design for a time-varies strategy revision rate to guarantee convergence.

Abstract

We study a multi-agent decision problem in population games, where agents select from multiple available strategies and continually revise their selections based on the payoffs associated with these strategies. Unlike conventional population game formulations, we consider a scenario where agents must estimate the payoffs through local measurements and communication with their neighbors. By employing task allocation games -- dynamic extensions of conventional population games -- we examine how errors in payoff estimation by individual agents affect the convergence of the strategy revision process. Our main contribution is an analysis of how estimation errors impact the convergence of the agents' strategy profile to equilibrium. Based on the analytical results, we propose a design for a time-varying strategy revision rate to guarantee convergence. Simulation studies illustrate how the proposed method for updating the revision rate facilitates convergence to equilibrium.
Paper Structure (14 sections, 2 theorems, 39 equations, 2 figures)

This paper contains 14 sections, 2 theorems, 39 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Preliminaries and Problem Description
  3. Task Allocation Game Model
  4. Learning Rule, Stochastic Alarm Clock, and Evolutionary Dynamics Model
  5. Payoff Vector Estimation
  6. Convergence Analysis and Revision Rate Update
  7. Convergence to Equilibrium State
  8. Revision Rate Update
  9. Simulations
  10. Conclusion
  11. Passivity of Game Model and EDM
  12. $\delta$-Antipassivity of Game Model \ref{['eq:task_allocation_games']}
  13. $\delta$-Passivity of EDM \ref{['eq:edm']}
  14. Proof of Theorem \ref{['thm:convergence']}

Key Result

Theorem 1

For a given revision rate $\lambda$, let $q_{\lambda}(t)$ and $x_{\lambda}(t)$ be the game and population states, respectively, determined by the feedback interconnection of the game model eq:task_allocation_games and EDM eq:edm_with_error_term. Under Assumptions assumption:function_F-assumption:est where $(q^\ast, x^\ast)$ is the unique equilibrium state of the closed loop system defined by eq:ta

Figures (2)

  • Figure 1: Trajectories of the game state derived by the Smith learning rule in the task allocation game described in Example \ref{['example:task_allocation_games']}, where the agents are estimating the payoff vector using \ref{['eq:consensus_estimation_rule']}.
  • Figure 2: Population state and game state trajectories when the rate of the Poisson alarm clock is updated according to the method described in Section \ref{['sec:methods']}. The trajectories are examined using three different parameter choices: (a) $\gamma=0.8, \tau=0.2$, (b) $\gamma=0.95, \tau=1.0$, and (c) $\gamma=0.99, \tau=1.4$.

Theorems & Definitions (7)

  • Example 1
  • Remark 1
  • Example 2
  • Example 3
  • Theorem 1
  • Lemma 1
  • proof