Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convergence of Payoff-Based Higher-Order Replicator Dynamics in Contractive Games

Hassan Abdelraouf, Vijay Gupta, Jeff S. Shamma

Abstract

We study the convergence properties of a payoff-based higher-order version of replicator dynamics, a widely studied model in evolutionary dynamics and game-theoretic learning, in contractive games. Recent work has introduced a control-theoretic perspective for analyzing the convergence of learning dynamics through passivity theory, leading to a classification of learning dynamics based on the passivity notion they satisfy, such as \textdelta-passivity, equilibrium-independent passivity, and incremental passivity. We leverage this framework for the study of higher-order replicator dynamics for contractive games, which form the complement of passive learning dynamics. Standard replicator dynamics can be represented as a cascade interconnection between an integrator and the softmax mapping. Payoff-based higher-order replicator dynamics include a linear time-invariant (LTI) system in parallel with the existing integrator. First, we show that if this added system is strictly passive and asymptotically stable, then the resulting learning dynamics converge locally to the Nash equilibrium in contractive games. Second, we establish global convergence properties using incremental stability analysis for the special case of symmetric matrix contractive games.

Convergence of Payoff-Based Higher-Order Replicator Dynamics in Contractive Games

Abstract

We study the convergence properties of a payoff-based higher-order version of replicator dynamics, a widely studied model in evolutionary dynamics and game-theoretic learning, in contractive games. Recent work has introduced a control-theoretic perspective for analyzing the convergence of learning dynamics through passivity theory, leading to a classification of learning dynamics based on the passivity notion they satisfy, such as \textdelta-passivity, equilibrium-independent passivity, and incremental passivity. We leverage this framework for the study of higher-order replicator dynamics for contractive games, which form the complement of passive learning dynamics. Standard replicator dynamics can be represented as a cascade interconnection between an integrator and the softmax mapping. Payoff-based higher-order replicator dynamics include a linear time-invariant (LTI) system in parallel with the existing integrator. First, we show that if this added system is strictly passive and asymptotically stable, then the resulting learning dynamics converge locally to the Nash equilibrium in contractive games. Second, we establish global convergence properties using incremental stability analysis for the special case of symmetric matrix contractive games.
Paper Structure (12 sections, 6 theorems, 32 equations, 6 figures)

This paper contains 12 sections, 6 theorems, 32 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Notations
  3. Preliminaries
  4. Replicator dynamics and its higher-order variants
  5. Population Games and Nash Stationarity
  6. Passivity and incremental stability
  7. Local Convergence of Higher-Order Replicator Dynamics in Contractive Games
  8. Global Convergence of Higher-order replicator dynamics in matrix contractive games
  9. Case 1: $g(s)$ passive.
  10. Case 2: $g(s)$ strictly passive.
  11. Conclusion
  12. References

Key Result

Theorem 1

If $F$ is continuously differentiable, then $F$ is contractive if and only if its Jacobian satisfies $z^\top \nabla F(x)z \le 0 \quad \forall x\in\Delta_n,\; z\in \mathcal{Z}.$

Figures (6)

  • Figure 1: Block diagram of the linearized local dynamics.
  • Figure 2: Local convergence of payoff-based higher-order replicator dynamics for $h(s)=\tfrac{2s+3}{s^2+3s+2}$ in the RPS game.
  • Figure 3: Implementation of generalized RD in a matrix game.
  • Figure 4: Block diagram representation of the linearized system.
  • Figure 5: Congestion network
  • ...and 1 more figures

Theorems & Definitions (18)

  • Definition 1: Nash Equilibrium
  • Definition 2: Contractive Games hofbauer2009stable
  • Theorem 1: Stable Games hofbauer2009stable
  • Definition 3: Nash Stationarity sandholm2010population
  • Lemma 1
  • Definition 4: Passivity
  • Definition 5: Incremental Stability forni2013differential
  • Lemma 2
  • proof
  • Theorem 2
  • ...and 8 more