Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Counterclockwise Dissipativity, Potential Games and Evolutionary Nash Equilibrium Learning

Nuno C. Martins, Jair Certório, Matthew S. Hankins

TL;DR

The paper tackles evolutionary Nash equilibrium learning in large populations with potentially dynamic payoff mechanisms. It introduces counterclockwise dissipativity (CCW) as a unifying framework that accommodates both imitation-based rules and continuous delta-passive rules through conic combinations, linking CCW payoff mechanisms to potential games for memoryless cases. The authors prove that CCW payoff mechanisms guarantee convergence of the population state to the Nash equilibria set of the associated stationary game for any hybrid learning rule within a convex cone, and they provide a numerical example illustrating convergence to a symmetric Nash equilibrium. This work broadens the applicability of passivity-based analysis in multi-agent learning, offering design guidance for payoff mechanisms that ensure evolutionary Nash equilibrium learning even when the exact learning rule is uncertain or mixed. Overall, CCW dissipativity provides a robust, broadly applicable framework that encompasses delta-passive, imitator-based, and approximated best-response behaviors in population games.

Abstract

We use system-theoretic passivity methods to study evolutionary Nash equilibria learning in large populations of agents engaged in strategic, non-cooperative interactions. The agents follow learning rules (rules for short) that capture their strategic preferences and a payoff mechanism ascribes payoffs to the available strategies. The population's aggregate strategic profile is the state of an associated evolutionary dynamical system. Evolutionary Nash equilibrium learning refers to the convergence of this state to the Nash equilibria set of the payoff mechanism. Most approaches consider memoryless payoff mechanisms, such as potential games. Recently, methods using $δ$-passivity and equilibrium independent passivity (EIP) have introduced dynamic payoff mechanisms. However, $δ$-passivity does not hold when agents follow rules exhibiting ``imitation" behavior, such as in replicator dynamics. Conversely, EIP applies to the replicator dynamics but not to $δ$-passive rules. We address this gap using counterclockwise dissipativity (CCW). First, we prove that continuous memoryless payoff mechanisms are CCW if and only if they are potential games. Subsequently, under (possibly dynamic) CCW payoff mechanisms, we establish evolutionary Nash equilibrium learning for any rule within a convex cone spanned by imitation rules and continuous $δ$-passive rules.

Counterclockwise Dissipativity, Potential Games and Evolutionary Nash Equilibrium Learning

TL;DR

The paper tackles evolutionary Nash equilibrium learning in large populations with potentially dynamic payoff mechanisms. It introduces counterclockwise dissipativity (CCW) as a unifying framework that accommodates both imitation-based rules and continuous delta-passive rules through conic combinations, linking CCW payoff mechanisms to potential games for memoryless cases. The authors prove that CCW payoff mechanisms guarantee convergence of the population state to the Nash equilibria set of the associated stationary game for any hybrid learning rule within a convex cone, and they provide a numerical example illustrating convergence to a symmetric Nash equilibrium. This work broadens the applicability of passivity-based analysis in multi-agent learning, offering design guidance for payoff mechanisms that ensure evolutionary Nash equilibrium learning even when the exact learning rule is uncertain or mixed. Overall, CCW dissipativity provides a robust, broadly applicable framework that encompasses delta-passive, imitator-based, and approximated best-response behaviors in population games.

Abstract

We use system-theoretic passivity methods to study evolutionary Nash equilibria learning in large populations of agents engaged in strategic, non-cooperative interactions. The agents follow learning rules (rules for short) that capture their strategic preferences and a payoff mechanism ascribes payoffs to the available strategies. The population's aggregate strategic profile is the state of an associated evolutionary dynamical system. Evolutionary Nash equilibrium learning refers to the convergence of this state to the Nash equilibria set of the payoff mechanism. Most approaches consider memoryless payoff mechanisms, such as potential games. Recently, methods using -passivity and equilibrium independent passivity (EIP) have introduced dynamic payoff mechanisms. However, -passivity does not hold when agents follow rules exhibiting ``imitation" behavior, such as in replicator dynamics. Conversely, EIP applies to the replicator dynamics but not to -passive rules. We address this gap using counterclockwise dissipativity (CCW). First, we prove that continuous memoryless payoff mechanisms are CCW if and only if they are potential games. Subsequently, under (possibly dynamic) CCW payoff mechanisms, we establish evolutionary Nash equilibrium learning for any rule within a convex cone spanned by imitation rules and continuous -passive rules.
Paper Structure (21 sections, 4 theorems, 26 equations, 2 figures)

This paper contains 21 sections, 4 theorems, 26 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Background On Passivity Approaches
  3. Main Objective
  4. Contributions And Limitations Of Our Approach
  5. Outline of the Paper
  6. Framework and Problem Formulation
  7. Learning Rules And Evolutionary Dynamics
  8. Payoff Mechanism And Solutions
  9. Games and Potential Games
  10. Linear Time Invariant Payoff Mechanism
  11. Positive Correlation And Hybrid Rules
  12. Tellegen And Universality Of Positive Correlation
  13. Canonical Rule Classes Satisfying (PC)
  14. Convex Cones, Hybrid Rules and Key Properties
  15. Counterclockwise Dissipativity And Main Results
  16. ...and 6 more sections

Key Result

Proposition 1

If $\mathcal{T}$ is a hybrid rule expressible as (eq:HybLearnRule), then $\mathcal{T}$ satisfies (PC) with the correlation function where $\wp^\mathrm{I}$, $\wp^\mathrm{CO}$, $\wp^\mathrm{EP}$ and $\tilde{\wp}$ are, respectively, the correlation functions for $\mathcal{T}^\mathrm{I}$, $\mathcal{T}^\mathrm{CO}$, $\mathcal{T}^\mathrm{EP}$ and $\tilde{\mathcal{T}}$. In addition, $\wp$ satisfies the

Figures (2)

  • Figure 1: Interconnection of (EDM) and payoff mechanism $\mathfrak{F}$.
  • Figure 2: Trajectories for the population state $x(t)$ converging to $\mathbb{NE}(\mathcal{F})$ under different initial conditions and learning rules, respectively, ${x(0)\in \{c,d,e,f\}}$ and $\mathcal{T}\in \{\mathcal{T}^{\mathrm{BNN}}, \mathcal{T}^{\mathrm{Smith}}, \mathcal{T}^{\mathrm{b}}\}$.

Theorems & Definitions (25)

  • Definition 1
  • Remark 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Remark 2
  • Definition 5
  • Example 1
  • Definition 6
  • Example 2
  • ...and 15 more