Approximate Feedback Nash Equilibria with Sparse Inter-Agent Dependencies

TL;DR

This work addresses the challenge of implementing feedback Nash equilibria in multi-agent dynamic games under sensing and communication constraints by introducing a regularized dynamic programming framework that induces sparsity in inter-agent dependencies via an adaptive group Lasso. For linear-quadratic games, the method preserves convexity and provides a bound on the deviation from standard Nash policies, with convergence of the regularized Riccati recursion to a neighborhood of the Nash solution. The approach extends to non-LQ games through iterative LQ approximations, and experimental results on multi-robot navigation and formation tasks demonstrate that sparse policies reduce communication requirements and improve robustness to noisy observations, often outperforming standard Nash equilibria. Overall, the framework offers a principled way to trade off sparsity and performance, with practical implications for resource-constrained multi-agent systems.

Abstract

Feedback Nash equilibrium strategies in multi-agent dynamic games require availability of all players' state information to compute control actions. However, in real-world scenarios, sensing and communication limitations between agents make full state feedback expensive or impractical, and such strategies can become fragile when state information from other agents is inaccurate. To this end, we propose a regularized dynamic programming approach for finding sparse feedback policies that selectively depend on the states of a subset of agents in dynamic games. The proposed approach solves convex adaptive group Lasso problems to compute sparse policies approximating Nash equilibrium solutions. We prove the regularized solutions' asymptotic convergence to a neighborhood of Nash equilibrium policies in linear-quadratic (LQ) games. Further, we extend the proposed approach to general non-LQ games via an iterative algorithm. Simulation results in multi-robot interaction scenarios show that the proposed approach effectively computes feedback policies with varying sparsity levels. When agents have noisy observations of other agents' states, simulation results indicate that the proposed regularized policies consistently achieve lower costs than standard Nash equilibrium policies by up to 77% for all interacting agents whose costs are coupled with other agents' states.

Figures (6)

• Figure 1: (a) Snapshot of an episode from the navigation game example. (b-c): Regularized and standard Nash equilibrium policy matrices, denoted as $\hat{P}_t$ in \ref{['eq:reg-problem']} and $P_t$ in \ref{['eq:system-equations']}. A policy matrix $P_t = [P_t^{1\top}, \hdots, P_t^{N\top}]^\top$, aggregated over players, maps all players' states to joint controls: $u_t = - P_t x_t - \alpha_t$. Each policy matrix is divided into $N \times N$ blocks.
• Figure 2: The number of nonzero blocks over time contained in policies of players 1-4 with different regularization levels.
• Figure 3: Snapshot of the proposed approach for an 8-player navigation game and corresponding policy matrices.
• Figure 4: (a) A three-player formation game: player 1 tracks a reference trajectory shown in grey while players 2 and 3 maintain a formation with respect to player 1. (b) Convergence of regularized policies with different $\lambda$.
• Figure 5: Costs of players 1-3 subtracted by standard Nash equilibrium costs in a formation game, averaged over 100 random initial players' positions, for varying noise and regularization levels. In subplots 2-3, the orange lines indicate the zero level set.

Theorems & Definitions (8)

• Definition 1
• Definition 2
• Remark 1
• Lemma 1
• Theorem 1
• Corollary 1
• Remark 2
• Definition 3