Characterizing the Convergence of Game Dynamics via Potentialness

TL;DR

The paper addresses how learning dynamics converge or fail to converge in multi-agent settings by introducing potentialness, a metric that quantifies how close a game is to being a potential game. It builds on Candogan et al.'s flow decomposition to separate a game's deviation flow into potential and harmonic components, and defines potentialness as the ratio of the potential-flow magnitude to the total flow magnitude. The authors develop a numerical framework to compute this metric and demonstrate that potentialness predicts both the existence of pure Nash equilibria and the convergence of no-regret dynamics across random, standard, and economically structured games, including Bayesian extensions. Empirically, potentialness tends to decrease with more agents or actions and higher potentialness correlates with higher convergence probability, with Bayesian Bayesian games showing increased potentialness and opportunities for pure Bayes-Nash equilibria. These insights offer a principled ex-ante predictor for learning outcomes in complex strategic environments and suggest directions for broader dynamical analyses and continuous-action extensions.

Abstract

Understanding the convergence landscape of multi-agent learning is a fundamental problem of great practical relevance in many applications of artificial intelligence and machine learning. While it is known that learning dynamics converge to Nash equilibrium in potential games, the behavior of dynamics in many important classes of games that do not admit a potential is poorly understood. To measure how ''close'' a game is to being potential, we consider a distance function, that we call ''potentialness'', and which relies on a strategic decomposition of games introduced by Candogan et al. (2011). We introduce a numerical framework enabling the computation of this metric, which we use to calculate the degree of ''potentialness'' in generic matrix games, as well as (non-generic) games that are important in economic applications, namely auctions and contests. Understanding learning in the latter games has become increasingly important due to the wide-spread automation of bidding and pricing with no-regret learning algorithms. We empirically show that potentialness decreases and concentrates with an increasing number of agents or actions; in addition, potentialness turns out to be a good predictor for the existence of pure Nash equilibria and the convergence of no-regret learning algorithms in matrix games. In particular, we observe that potentialness is very low for complete-information models of the all-pay auction where no pure Nash equilibrium exists, and much higher for Tullock contests, first-, and second-price auctions, explaining the success of learning in the latter. In the incomplete-information version of the all-pay auction, a pure Bayes-Nash equilibrium exists and it can be learned with gradient-based algorithms. Potentialness nicely characterizes these differences to the complete-information version.

Figures (10)

• Figure 1: Decomposition of the Shapley game.
• Figure 2: Average runtime for computing potentialness over 100 runs, with precomputed matrices.
• Figure 3: Empirical distribution of potentialness in $10^6$ randomly generated games for each considered setting. Increasing the size of the games reduces the variance and the mean of the distribution.
• Figure 4: Relationship between potentialness and existence of SPNE across different settings. Vertical axis: empirical probability of SPNE existence as a function of the setting. Horizontal axis: Potentialness values. The width of the horizontal lines represents the observed range of potentialness values for each setting, corresponding to the support of the densities shown in \ref{['fig:distribution_potentialness']}. The dot on each line indicates the average potentialness for that setting. As game size increases, mean potentialness decreases, and the probability of SPNE existence approaches $1 - 1/e$ (dotted horizontal line).
• Figure 5: Relationship between potentialness and existence of SPNE within the same setting. Vertical axis: empirical probability of SPNE existence as a function of potentialness. Horizontal axis: potentialness values. For each setting, a higher potentialness increases the likelihood that a game has at least one SPNE.

Theorems & Definitions (20)

• Definition 1: Normal-form game
• Definition 2: Mixed strategy
• Definition 3: Nash equilibrium
• Definition 4: Potential game
• Definition 5: Deviation map
• Definition 6: Gradient map
• Proposition 1
• proof
• Theorem : candogan2011flows --- Combinatorial Hodge decomposition of feasible flows
• Definition 7: Potentialness