A system level approach to generalised feedback Nash equilibrium seeking in partially-observed games

TL;DR

The paper addresses noncooperative dynamic games with linear, stochastic, partially observed dynamics and seeks generalized feedback Nash equilibria (GFNE) using System Level Synthesis (SLS). By reparametrising output‑feedback policies as system level responses, the authors convert the dynamic game into a finite‑dimensional static game and apply monotone‑operator theory with forward‑backward splitting to compute a variational GFNE (vGFNE) while enforcing stability, operational, and communication constraints. They provide convergence guarantees and practical numerics, including FIR truncation and chance‑constrained robustness, enabling online policy learning that can run alongside system operation. The illustrative power‑grid example demonstrates that agents learn stabilising GFNE policies under delays and partial observation, achieving near‑GFNE performance while respecting constraints. This approach offers a scalable, decentralized learning framework for GFNE in large‑scale cyber‑physical systems with partial observability.

Abstract

This work proposes an algorithm for seeking generalised feedback Nash equilibria (GFNE) in noncooperative dynamic games. The focus is on cyber-physical systems with dynamics which are linear, stochastic, potentially unstable, and partially observed. We employ System Level Synthesis (SLS) to reformulate the problem as the search for an equilibrium profile of closed-loop responses to noise, which can then be used to reconstruct a stabilising output-feedback policy. Under this setup, we leverage monotone operator theory to design a GFNE-seeking algorithm capable to enforce closed-loop stability, operational constraints, and communication constraints onto the control policies. This algorithm is amenable to numerical implementation and we provide conditions for its convergence. We demonstrate our approach in a simulated experiment on the noncooperative stabilisation of a decentralised power-grid.

Figures (5)

• Figure 1: Feedback structure for policies ${\hat{K}}^p = {\hat{\Phi}_{uy}}^p - {\hat{\Phi}_{ux}}^p {\hat{\Phi}_{xx}}^{-1} {\hat{\Phi}_{xy}} = {\tilde{\Phi}_{uy}}^p + {\tilde{\Phi}_{ux}}^p(zI - {\tilde{\Phi}_{xx}})^{-1}{\tilde{\Phi}_{xy}}$, $p \in \mathcal{P}$, according to Eq. \ref{['eq: SLP_ControlImplementation']}.
• Figure 2: Power-grid: Schematic of the interconnected network (left) and the interactions within each $p$-th subsystem (right).
• Figure 3: Power-grid: Convergence of the vGFNE-seeking routine.
• Figure 4: Power-grid: Closed-loop (left panels) and open-loop (right panels) responses in terms of frequency loads $u^p$, phase angle measurements $y^p_t$, and pairwise buses $\kappa^{p,\tilde{p}}(\theta^p_t - \theta^{\tilde{p}}_t)$, for all $p,\tilde{p} \in \mathcal{P}$. The signals from each subsystem are plotted superimposed. The shaded region indicates the unsafe operation region of the network channels.
• Figure 5: Power-grid: Closed-loop impulse response in terms of measurements $y^p$ and control inputs $u^p$, for each individual subsystem $p \in \mathcal{P}$.

Theorems & Definitions (12)

• Definition 1
• Definition 2
• Theorem 1
• proof
• Definition 3
• Theorem 2: System level parametrisation
• proof
• Corollary 2.1
• Remark 1
• Theorem 3