Optimally Installing Strict Equilibria
Jeremy McMahan, Young Wu, Yudong Chen, Xiaojin Zhu, Qiaomin Xie
TL;DR
This work develops a reward-design framework to install a target behavior as a strict equilibrium across DSE, NE, CE, and CCE, including their Markov-perfect variants, in known-transition Markov games. It provides exact feasibility characterizations for strictly installable policies in normal-form games and extends these to stage-wise and Markov-game settings, yielding polynomial-time algorithms to verify installability and construct feasible rewards. The authors introduce LP-based methods to solve the optimal reward-design problem under a reward bound and discuss robustness under bounded rationality via epsilon-strict equilibria, including detailed deviation-class analyses. Key contributions include complete CE/CCE installability characterizations, epsilon-strictness bounds, and scalable algorithms for both normal-form and Markov-game settings, enabling practical deployment of strong, predictable equilibria in sequential multi-agent environments. The results have practical impact for safe traffic control, resource allocation, and other domains where enforcing a desired compliant behavior is critical while keeping changes within realistic constraints.
Abstract
In this work, we develop a reward design framework for installing a desired behavior as a strict equilibrium across standard solution concepts: dominant strategy equilibrium, Nash equilibrium, correlated equilibrium, and coarse correlated equilibrium. We also extend our framework to capture the Markov-perfect equivalents of each solution concept. Central to our framework is a comprehensive mathematical characterization of strictly installable, based on the desired solution concept and the behavior's structure. These characterizations lead to efficient iterative algorithms, which we generalize to handle optimization objectives through linear programming. Finally, we explore how our results generalize to bounded rational agents.