Anytime-Constrained Equilibria in Polynomial Time
Jeremy McMahan
TL;DR
This work introduces anytime constraints in constrained Markov games by defining ACE and related equilibrium concepts, and it establishes both hardness and algorithmic results. It reduces feasibility in acMGs to action-constrained MGs, enabling backward-induction computation of ACE via LP-based solutions for constrained stage games. The authors provide a fixed-parameter tractable algorithm for subgame-perfect ACE when cost precision is small and a polynomial-time approximation for ACE when costs are large but bounded, with guarantees tied to cost granularity $\ell$ and maximum costs. A realizability-graph framework and an AND/OR feasibility tree underpin feasibility analysis, while a rounding-based approximation approach yields scalable, provably near-feasible equilibria. The results culminate in a robust theory of action-constrained MGs, offering practical pathways for efficient ACE computation in multi-agent constrained settings.
Abstract
We extend anytime constraints to the Markov game setting and the corresponding solution concept of an anytime-constrained equilibrium (ACE). Then, we present a comprehensive theory of anytime-constrained equilibria that includes (1) a computational characterization of feasible policies, (2) a fixed-parameter tractable algorithm for computing ACE, and (3) a polynomial-time algorithm for approximately computing ACE. Since computing a feasible policy is NP-hard even for two-player zero-sum games, our approximation guarantees are optimal so long as $P \neq NP$. We also develop the first theory of efficient computation for action-constrained Markov games, which may be of independent interest.