Game Theory with Simulation in the Presence of Unpredictable Randomisation

TL;DR

The paper studies costly simulation of AI agents in two-player games, distinguishing mixed-strategy from pure-strategy simulation under unpredictable randomness. It formalizes mixed-strategy simulation games, proves a finite reduction and NP-hardness results for determining when simulation yields Pareto-improving equilibria, and identifies classes of games where mixed-simulation improves welfare, notably generalised partial-trust games, trust-plus-coordination, and privacy-aware settings. The results show that mixed-strategy simulation can either fail to enhance cooperation (as in the standard TG when the simulatee observes the base action) or enable welfare gains when players face trust and coordination challenges or privacy needs. These findings offer guidance for designing interventions and mechanisms to promote cooperation in AI-agent interactions, while highlighting computational barriers in predicting simulation’s effects in general games.

Abstract

AI agents will be predictable in certain ways that traditional agents are not. Where and how can we leverage this predictability in order to improve social welfare? We study this question in a game-theoretic setting where one agent can pay a fixed cost to simulate the other in order to learn its mixed strategy. As a negative result, we prove that, in contrast to prior work on pure-strategy simulation, enabling mixed-strategy simulation may no longer lead to improved outcomes for both players in all so-called "generalised trust games". In fact, mixed-strategy simulation does not help in any game where the simulatee's action can depend on that of the simulator. We also show that, in general, deciding whether simulation introduces Pareto-improving Nash equilibria in a given game is NP-hard. As positive results, we establish that mixed-strategy simulation can improve social welfare if the simulator has the option to scale their level of trust, if the players face challenges with both trust and coordination, or if maintaining some level of privacy is essential for enabling cooperation.

Figures (6)

• Figure 1: Trust game $\texttt{TG}$ and its partial-trust extension $\texttt{PTG}$.
• Figure 2: Top: An illustration of a generalised partial-trust game ${\mathcal{G}}$ from \ref{['def:gPTG']}. Middle: Examples of strategies that would invalidate the technical conditions in the definition if we added them to ${\mathcal{G}}$. $\textnormal{T}'_1$ fails (4a), since it does not have a unique value $u_1(\textnormal{T}, \textnormal{C})$. $\textnormal{T}'_2$ fails the requirement (4b), that any increase in $u_1(\textnormal{T}, \textnormal{C})$ -- here caused by going from $\textnormal{T}_2$ to $\textnormal{T}'_2$ -- must also increase $u_2(\textnormal{T}, \textnormal{C})$ (and $u_2(\textnormal{T}, \textnormal{D})$, and decrease $u_1 (\textnormal{T}, \textnormal{D})$). $\textnormal{T}'_{1.5}$ fails (5), since $\textnormal{T}'_{1.5}$ yields the same $u_1$ as the convex combination $\frac{1}{2} \cdot \textnormal{T}_1 + \frac{1}{2} \cdot \textnormal{T}_2$ without also having the same $u_2$. Bottom:$\textnormal{T}_{1.9}$ is an example of a strategy that would not invalidate any of the technical conditions from the definition, but adding it to ${\mathcal{G}}$ would break the assumption of "sufficiently high $u_2(\textnormal{FT}, \textnormal{C})$" that is required for \ref{['thm:generalised_PTG']}.
• Figure 3: Trust-and-coordination game, where coordinating on a joint action $(a_1^k, a_2^k)$ leads the players to a trust subgame.
• Figure 5: A concrete and a parameterised version of the Partial-Trust Game from \ref{['fig:PTG']}. The names of the constants are meant as mnemonics for (fully- and partially-) Bad, Neutral, Good, and Awesome. Correspondingly, we assume that $\textnormal{B}^\textnormal{F}_1 < \textnormal{B}^\textnormal{P}_1 < \textnormal{N}_1 < \textnormal{G}^\textnormal{P}_1 < \textnormal{G}^\textnormal{F}_1$ and $\textnormal{N}_2 < \textnormal{G}^\textnormal{P}_2 < \textnormal{A}^\textnormal{P}_2$, $\textnormal{N}_2 < \textnormal{G}^\textnormal{F}_2 < \textnormal{A}^\textnormal{F}_2$, $\textnormal{G}^\textnormal{P}_2 < \textnormal{G}^\textnormal{F}_2$, and $\textnormal{A}^\textnormal{P}_2 < \textnormal{A}^\textnormal{F}_2$ (though the last one is not strictly necessary). (The relationship between $\textnormal{G}^\textnormal{F}_2$ and $\textnormal{A}^\textnormal{P}_2$ is not important.)
• Figure 6: A generalised form of a trust-and-coordination game. The general version of the game has $n+1$ actions for each player, with the joint action $(a_1^k, a_2^k)$ leading into the subgame ${\mathcal{G}}_k$. We assume that $\textnormal{H}^k_1 < \textnormal{B}_1 < \textnormal{N}^k_1 < \textnormal{G}^k_1$ and $\textnormal{H}^k_2 < \textnormal{B}_2 < \textnormal{G}^k_1 < \textnormal{A}^k_1$ holds for every $k$ and $\epsilon > 0$ is small enough to not affect this ordering. Note that each of the subgames ${\mathcal{G}}_k$ is a trust game, with equilibrium $(\textnormal{Walk Out}, \textnormal{Defect})$. As a result, the only NE of the large game is for each of the players to take the $\textnormal{Opt Out}$.

Theorems & Definitions (42)

• Definition 3.1: Pure- and mixed-strategy simulation
• Lemma 3.1
• Remark 3.2
• Proposition 3.2: Reduction to a finite strategy space
• Proposition 4.0: Upper bound on solving $\msimgame$
• Theorem 4.0: Determining whether simulation helps is hard
• Theorem 5.0: Simulating a perfectly informed player
• Definition 5.1: Generalised Partial-Trust Game
• Lemma 5.1
• Theorem 5.1: Simulation helps with partial trust