Efficiently Solving Mixed-Hierarchy Games with Quasi-Policy Approximations

TL;DR

This work addresses the challenge of computing equilibria in N-robot mixed Nash-Stackelberg games with forest-structured information, where leaders influence followers and branches interact via Nash behavior. It derives $KKT$ conditions that require high-order derivatives of followers' policies, then introduces a quasi-policy approximation to keep only first-order policy gradients and an inexact Newton method to solve the resulting nonlinear, equality-constrained system, with local exponential convergence guarantees. The authors provide a principled construction of approximate optimality conditions for nonlinear, forest-structured mixed-hierarchy games, an efficient solver, convergence analysis, and an open-source Julia library MixedHierarchyGames.jl, validated by real-time performance in merging and pursuit-guard-target scenarios. This approach enables scalable, real-time multi-robot coordination under asymmetric information and mixed decision-making structures, bridging Nash and Stackelberg reasoning in nonlinear settings.

Abstract

Multi-robot coordination often exhibits hierarchical structure, with some robots' decisions depending on the planned behaviors of others. While game theory provides a principled framework for such interactions, existing solvers struggle to handle mixed information structures that combine simultaneous (Nash) and hierarchical (Stackelberg) decision-making. We study N-robot forest-structured mixed-hierarchy games, in which each robot acts as a Stackelberg leader over its subtree while robots in different branches interact via Nash equilibria. We derive the Karush-Kuhn-Tucker (KKT) first-order optimality conditions for this class of games and show that they involve increasingly high-order derivatives of robots' best-response policies as the hierarchy depth grows, rendering a direct solution intractable. To overcome this challenge, we introduce a quasi-policy approximation that removes higher-order policy derivatives and develop an inexact Newton method for efficiently solving the resulting approximated KKT systems. We prove local exponential convergence of the proposed algorithm for games with non-quadratic objectives and nonlinear constraints. The approach is implemented in a highly optimized Julia library (MixedHierarchyGames.jl) and evaluated in simulated experiments, demonstrating real-time convergence for complex mixed-hierarchy information structures.

Figures (2)

• Figure 1: We experiment on four hierarchical information structures on a convoy merging scenario and report positions over time and distances between vehicles for each configuration. In \ref{['subfig:merging-mixed-a']}, vehicle 1 leads every other agent and vehicle 2 leads vehicle 4. Vehicles 3 and (2,4) are negotiating the merge simultaneously. In \ref{['subfig:merging-nash']}, vehicles play a pure Nash game. Experiments for mixed (B) and Stackelberg chain hierarchy structures are shown in \ref{['fig:merging-experiments']}.
• Figure 2: