Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Feedback and Open-Loop Nash Equilibria for LQ Infinite-Horizon Discrete-Time Dynamic Games

A. Monti, B. Nortmann, T. Mylvaganam, M. Sassano

TL;DR

This work studies infinite-horizon, discrete-time linear-quadratic dynamic games with two players, focusing on Feedback-Nash Equilibria (F-NE) and Open-Loop-Nash Equilibria (OL-NE). It derives a complete DP-based characterization of F-NE via coupled algebraic equations and a stabilizability condition, while introducing a novel PMP-informed analysis to characterize OL-NE through an auxiliary single-player problem with exogenous input and a Hamiltonian framework. A key result is the OL-NE feedback-synthesis: under a spectral assumption on a constructed Hamiltonian matrix, the OL-NE can be implemented as a linear state feedback with gains found from a pair of asymmetric Riccati equations. Numerical examples show that F-NE and OL-NE generally differ and that OL-NE feedback synthesis is achievable under the proposed conditions, illustrating practical distinctions between equilibrium concepts. The findings bridge DP and PMP approaches for discrete-time LQ games and provide a foundation for data-driven or distributed implementations in multi-agent control systems.

Abstract

We consider dynamic games defined over an infinite horizon, characterized by linear, discrete-time dynamics and quadratic cost functionals. Considering such linear-quadratic (LQ) dynamic games, we focus on their solutions in terms Nash equilibrium strategies. Both Feedback (F-NE) and Open-Loop (OL-NE) Nash equilibrium solutions are considered. The contributions of the paper are threefold. First, our detailed study reveals some interesting structural insights in relation to F-NE solutions. Second, as a stepping stone towards our consideration of OL-NE strategies, we consider a specific infinite-horizon discrete-time (single-player) optimal control problem, wherein the dynamics are influenced by a known exogenous input and draw connections between its solution obtained via Dynamic Programming and Pontryagin's Minimum Principle. Finally, we exploit the latter result to provide a characterization of OL-NE strategies of the class of infinite-horizon dynamic games. The results and key observations made throughout the paper are illustrated via a numerical example.

Feedback and Open-Loop Nash Equilibria for LQ Infinite-Horizon Discrete-Time Dynamic Games

TL;DR

This work studies infinite-horizon, discrete-time linear-quadratic dynamic games with two players, focusing on Feedback-Nash Equilibria (F-NE) and Open-Loop-Nash Equilibria (OL-NE). It derives a complete DP-based characterization of F-NE via coupled algebraic equations and a stabilizability condition, while introducing a novel PMP-informed analysis to characterize OL-NE through an auxiliary single-player problem with exogenous input and a Hamiltonian framework. A key result is the OL-NE feedback-synthesis: under a spectral assumption on a constructed Hamiltonian matrix, the OL-NE can be implemented as a linear state feedback with gains found from a pair of asymmetric Riccati equations. Numerical examples show that F-NE and OL-NE generally differ and that OL-NE feedback synthesis is achievable under the proposed conditions, illustrating practical distinctions between equilibrium concepts. The findings bridge DP and PMP approaches for discrete-time LQ games and provide a foundation for data-driven or distributed implementations in multi-agent control systems.

Abstract

We consider dynamic games defined over an infinite horizon, characterized by linear, discrete-time dynamics and quadratic cost functionals. Considering such linear-quadratic (LQ) dynamic games, we focus on their solutions in terms Nash equilibrium strategies. Both Feedback (F-NE) and Open-Loop (OL-NE) Nash equilibrium solutions are considered. The contributions of the paper are threefold. First, our detailed study reveals some interesting structural insights in relation to F-NE solutions. Second, as a stepping stone towards our consideration of OL-NE strategies, we consider a specific infinite-horizon discrete-time (single-player) optimal control problem, wherein the dynamics are influenced by a known exogenous input and draw connections between its solution obtained via Dynamic Programming and Pontryagin's Minimum Principle. Finally, we exploit the latter result to provide a characterization of OL-NE strategies of the class of infinite-horizon dynamic games. The results and key observations made throughout the paper are illustrated via a numerical example.
Paper Structure (8 sections, 6 theorems, 54 equations, 6 figures)

This paper contains 8 sections, 6 theorems, 54 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Problem statement and preliminaries
  3. F-NE for infinite-horizon dynamic games
  4. OL-NE for infinite-horizon dynamic games
  5. Infinite-horizon affine-quadratic optimal control
  6. OL-NE via Riccati equations
  7. Numerical simulations
  8. Conclusions and further extensions

Key Result

Theorem 3.2

\newlabelthm:F_NE0 Consider the system eq:lti and the cost functionals eq:cost_J_i, for $i = 1,2$. Then the pair of control laws for $i = 1,2$, constitutes a F-NE if and only if $K_i \in \mathbb{R}^{m_i \times n}$ are such that and solve, together with some $P_i \in \mathbb{R}^{n \times n}$, $P_i = P_i^{\top} \succeq 0$, $i = 1,2$, the matrix equations and Furthermore, the F-NE is such that $

Figures (6)

  • Figure 1: Time histories $x(k)$ corresponding to \ref{['eq:lti']} in closed loop with the feedback synthesis of the OL-NE strategies (blue) and with the F-NE strategies (red).
  • Figure 2: Time histories of $u_1(k)$ (solid lines) and $u_2(k)$ (dashed lines) corresponding to the OL-NE (blue) and the F-NE (red) strategies.
  • Figure 3: Graph of $J_1$ (blue line) corresponding to different strategies $u_1(k) = K_1x(k)$ with $u_2(k) = K_2^{OL} x(k)$. The vertical dotted line (black) corresponds to the gain $K_1^{OL}$ while the vertical dashed line (red) represents $\arg \min_{K_1} J_1(x_0,K_1x(k), K_2^{OL}x(k))$.
  • Figure 4: Graph of $J_2$ (blue line) corresponding to different strategies $u_2(k) = K_2x(k)$ with $u_1(k) = K_1^{OL} x(k)$. The vertical dotted line (black) corresponds to the gain $K_2^{OL}$ while the vertical dashed line (red) represents $\arg \min_{K_2} J_2(x_0, K_1^{OL}x(k), K_2x(k))$.
  • Figure 5: Graph of $J_1$ (blue line) corresponding to different strategies $u_1(k) = K_1x(k)$ with $u_2(k) = K_2^{OL}\left(A_{cl}^{OL}\right)^{k}x(0)$. The vertical dotted line (black) corresponds to the gain $K_1^{OL}$ and coincides with the vertical dashed line (red) representing $\arg \min_{K_1} J_2(x_0,K_1x(k), K_2^{OL}(A_{cl}^{OL})^kx(0))$.
  • ...and 1 more figures

Theorems & Definitions (16)

  • Definition 2.1
  • Definition 2.2
  • Remark 2.3
  • Theorem 3.2
  • Corollary 3.3
  • Remark 3.4
  • Lemma 4.2
  • Remark 4.3
  • Lemma 4.4
  • Remark 4.5
  • ...and 6 more