A Coupled Optimization Framework for Correlated Equilibria in Normal-Form Game
Sarah H. Q. Li, Yue Yu, Florian Dörfler, John Lygeros
TL;DR
The paper addresses the absence of a coupled optimization formulation for correlated equilibria in normal-form games by introducing an unnormalized game that lifts player strategies to unnormalized measures over the joint action space and showing that fully mixed generalized Nash equilibria of this lifted game correspond to correlated equilibria of the original game. It then introduces entropy regularization, deriving a closed-form generalized Nash equilibrium whose induced correlated strategy is an ε-correlated equilibrium of the normal-form game, with ε controlled by the regularization parameters. Empirical results on random finite games demonstrate both the sub-optimality bounds and substantial computational gains compared to direct CE polytope methods, while preserving the ability to approximate CE via a gradient-based optimization framework. This work lays groundwork for scalable, gradient-based multi-agent learning methods that target correlated equilibria by exploiting an optimization structure in the CE geometry.
Abstract
In competitive multi-player interactions, simultaneous optimality is a key requirement for establishing strategic equilibria. This property is explicit when the game-theoretic equilibrium is the simultaneously optimal solution of coupled optimization problems. However, no such optimization problems exist for the correlated equilibrium, a strategic equilibrium where the players can correlate their actions. We address the lack of a coupled optimization framework for the correlated equilibrium by introducing an {unnormalized game} -- an extension of normal-form games in which the player strategies are lifted to unnormalized measures over the joint actions. We show that the set of fully mixed generalized Nash equilibria of this unnormalized game is a subset of the correlated equilibrium of the normal-form game. Furthermore, we introduce an entropy regularization to the unnormalized game and prove that the entropy-regularized generalized Nash equilibrium is a sub-optimal correlated equilibrium of the normal form game where the degree of sub-optimality depends on the magnitude of regularization. We prove that the entropy-regularized unnormalized game has a closed-form solution, and empirically verify its computational efficacy at approximating the correlated equilibrium of normal-form games.