Solving Infinite-Player Games with Player-to-Strategy Networks

TL;DR

This work tackles solving games with infinitely many players by introducing a Player-to-Strategy Network (P2SN) that maps each player to a strategy and a Shared-Parameter Simultaneous Gradient (SPSG) algorithm to train the network toward an approximate Nash equilibrium. The approach generalizes multiagent gradient dynamics to an infinite-player setting using a single parameter set shared across all players, with Fourier-feature embeddings to capture high-frequency patterns in low-dimensional player spaces. Experiments across Ising-type, Cournot, and crowding-like games demonstrate convergence to near-NE profiles and decreasing exploitability metrics, even when utilities are discontinuous or actions are continuous and high-dimensional. The method extends equilibrium computation to infinite-population settings and offers a practical tool for analyzing and influencing large-scale multiagent systems and mechanism design.

Abstract

We present a new approach to solving games with a countably or uncountably infinite number of players. Such games are often used to model multiagent systems with a large number of agents. The latter are frequently encountered in economics, financial markets, crowd dynamics, congestion analysis, epidemiology, and population ecology, among other fields. Our two primary contributions are as follows. First, we present a way to represent strategy profiles for an infinite number of players, which we name a Player-to-Strategy Network (P2SN). Such a network maps players to strategies, and exploits the generalization capabilities of neural networks to learn across an infinite number of inputs (players) simultaneously. Second, we present an algorithm, which we name Shared-Parameter Simultaneous Gradient (SPSG), for training such a network, with the goal of finding an approximate Nash equilibrium. This algorithm generalizes simultaneous gradient ascent and its variants, which are classical equilibrium-seeking dynamics used for multiagent reinforcement learning. We test our approach on infinite-player games and observe its convergence to approximate Nash equilibria. Our method can handle games with infinitely many states, infinitely many players, infinitely many actions (and mixed strategies on them), and discontinuous utility functions.

Figures (17)

• Figure 1: High-level structure of the P2SN.
• Figure 2: Strategy profile for 1D Ising game.
• Figure 3: Regrets for 1D Ising game.
• Figure 4: Bias field for 2D Ising game.
• Figure 5: Strategy profile for 2D Ising game.