On Linear Quadratic Potential Games

TL;DR

The paper addresses when finite-horizon non-cooperative linear-quadratic games admit a potential function, revealing that full-state feedback yields a highly restricted potential class aligned with identical-interest games, and motivating a shift to decoupled dynamics and information structures. It develops necessary and sufficient conditions for LQ potentiality (Lemma 7), derives a broader potential class under decoupled dynamics (Theorem Th:CL_po) with an explicit potential function and guaranteed Nash equilibrium, and analyzes computational challenges including state correlations that affect best responses. A policy-gradient approach is shown to converge to stationary points of the potential function, with simulations demonstrating consistent convergence in multi-agent scenarios. The results provide a framework for understanding learning dynamics in multi-agent control under structured information, highlighting both theoretical insights and practical challenges for computing equilibria in decoupled LQ potential games.

Abstract

Our paper addresses characterizing conditions for a linear quadratic (LQ) game to be a potential game. The desired properties of potential games in finite action settings, such as convergence of learning dynamics to Nash equilibria, and the challenges of learning Nash equilibria in continuous state and action settings motivate us to characterize LQ potential games. Our first contribution is to show that the set of LQ games with full-state feedback that are potential games is very limited, essentially differing only slightly from an identical interest game. Given this finding, we restrict the class of LQ games to those with decoupled dynamics and decoupled state information structure. For this subclass, we show that the set of potential games strictly includes non-identical interest games and characterize conditions for the LQ games in this subclass to be potential. We further derive their corresponding potential function and prove the existence of a Nash equilibrium. Meanwhile, we highlight the challenges in the characterization and computation of Nash equilibrium for this class of potential LQ games.

Figures (1)

• Figure 1: Convergence of policy gradient with the update rule in equation (\ref{['eq:gradient']}) to a stationary point $\tilde{k}$ for 20 simulations. In these figures, $m$ indicates the iteration number, and the abscissa and the ordinate are on a logarithmic scale.