First-order (coarse) correlated equilibria in non-concave games
Mete Şeref Ahunbay
TL;DR
This work develops a first-order analogue of correlated equilibria for smooth, possibly non-concave games by introducing epsilon-local CE and epsilon-stationary CE with respect to vector-field deviations, including a coarse variant tied to gradient fields. It shows that online projected gradient ascent (with identical step sizes) universally approximates these equilibria in self-play and yields robust time-average guarantees under regularity on action sets; moreover, a continuous-curve construction connects discrete PGA updates to continuous gradient dynamics, enabling Lyapunov-function–style dual guarantees. The authors establish tractable approximability results for finite families of tangent vector fields (e.g., affine-linear tangent fields) and relate the primal-dual bounds to classical price-of-anarchy frameworks, including equivalences to coarse CE, CE, and smoothness in normal-form games. They also characterize when adversarial deviations can or cannot be handled (through tangency/well-tangency) and discuss extensions via Lagrangian Hedging, stochastic settings, and proximal regret notions. The framework unifies several equilibrium notions, provides computationally appealing procedures to approximate first-order CE notions, and offers a principled lens (via Lyapunov functions) to reason about time-average outcomes of gradient-based learning in non-concave games, with implications for learning dynamics, auctions, and multi-agent systems.
Abstract
We investigate first-order notions of correlated equilibria in smooth games, in which players do not incur any regret against small modifications of their actions prescribed by some vector field. We define two such notions, based on local deviations and on stationarity of the distribution, and identify the notion of coarseness as the setting where the strategy modifications are prescribed by gradient fields. For coarse equilibria, we prove that online (projected) gradient ascent has a universal approximation property for both variants of equilibrium; in the self-play setting, every differentiable function induces an equilibrium constraint, the approximation error of which depends on the modulus of continuity and magnitude of the gradient. In the adversarial setting, we instead obtain a characterisation of regret guarantees against continuous strategy modifications satisfied by projected gradient ascent; these are precisely deviations induced by gradient fields tangent to the action set. We also provide a generalisation of the Lagrangian Hedging framework, which identifies a novel refinement of correlated equilibrium which is tractable to approximate. We then study the primal-dual framework to our notion of first-order equilibria. For coarse equilibria defined by a family of functions, we find that a dual bound on the worst-case expectation of a performance metric takes the form of a generalised Lyapunov function for the dynamics of the game. Specifically, usual primal-dual price of anarchy analysis for coarse correlated equilibria as well as the smoothness framework of Roughgarden are both equivalent to a problem of general Lyapunov function estimation. For non-coarse equilibria, we instead observe that price of anarchy problems are dual to a vector field fit problem for the gradient dynamics of the game.