Passivity, No-Regret, and Convergent Learning in Contractive Games

TL;DR

The paper analyzes how passivity properties of continuous-time learning dynamics constrain regret and ensure convergence to Nash equilibria in contractive population games. It develops a framework where a learning model's passivity from the payoff to strategy deviation yields finite regret, proves this for direct-projection and various higher-order dynamics, and extends to equilibrium-independent passivity to relate regret and stability. Through a passivity-based classification (incremental, delta-, and EI-passivity) and a Lyapunov-based convergence argument, the work demonstrates global convergence to the unique Nash equilibrium in strictly contractive games and identifies fragility under payoff perturbations. These results provide principled guidance for designing learning dynamics with guaranteed no-regret properties and robust convergence in competitive environments.

Abstract

We investigate the interplay between passivity, no-regret, and convergence in contractive games for various learning dynamic models and their higher-order variants. Our setting is continuous time. Building on prior work for replicator dynamics, we show that if learning dynamics satisfy a passivity condition between the payoff vector and the difference between its evolving strategy and any fixed strategy, then it achieves finite regret. We then establish that the passivity condition holds for various learning dynamics and their higher-order variants. Consequentially, the higher-order variants can achieve convergence to Nash equilibrium in cases where their standard order counterparts cannot, while maintaining a finite regret property. We provide numerical examples to illustrate the lack of finite regret of different evolutionary dynamic models that violate the passivity property. We also examine the fragility of the finite regret property in the case of perturbed learning dynamics. Continuing with passivity, we establish another connection between finite regret and passivity, but with the related equilibrium-independent passivity property. Finally, we present a passivity-based classification of dynamic models according to the various passivity notions they satisfy, namely, incremental passivity, $δ$-passivity, and equilibrium-independent passivity. This passivity-based classification provides a framework to analyze the convergence of learning dynamic models in contractive games.

Figures (6)

• Figure 1: (SHO-FTRL) dynamic model
• Figure 2: (SHO-DP) dynamic model
• Figure 3: BNN, Smith, Logit, (TD) dynamics performance in the environment introduced in Example \ref{['ex: no regret counter example']}
• Figure 4: The performance of replicator dynamics with latency in the environment introduced in Example \ref{['ex: RD_latency']}
• Figure 5: Passivity-based classification of learning dynamic models

Theorems & Definitions (32)

• Remark 1
• Remark 2
• Definition 1: Nash Equilibrium
• Definition 2: Contractive Games hofbauer2009stable
• Theorem 1: Passivity and No-regret cheung2021online
• Theorem 2
• proof
• Proposition 1
• proof
• Theorem 3