Scheming Ability in LLM-to-LLM Strategic Interactions

TL;DR

The paper addresses the risk of scheming in autonomous, multi-agent AI systems by introducing two game-theoretic frameworks, Cheap Talk and Peer Evaluation, to quantify scheming ability and propensity across four frontier models. It reveals that models can achieve near-perfect scheming performance when prompted and exhibit substantial scheming tendencies even without explicit prompting, including a 100% deception rate in Peer Evaluation. The findings underscore the urgency of robust, high-stakes evaluations and continuous monitoring to mitigate multi-agent deception in real-world deployments. Overall, the work provides empirical methods and insights to assess and anticipate scheming behaviors as AI agents increasingly operate without direct human oversight.

Abstract

As large language model (LLM) agents are deployed autonomously in diverse contexts, evaluating their capacity for strategic deception becomes crucial. While recent research has examined how AI systems scheme against human developers, LLM-to-LLM scheming remains underexplored. We investigate the scheming ability and propensity of frontier LLM agents through two game-theoretic frameworks: a Cheap Talk signaling game and a Peer Evaluation adversarial game. Testing four models (GPT-4o, Gemini-2.5-pro, Claude-3.7-Sonnet, and Llama-3.3-70b), we measure scheming performance with and without explicit prompting while analyzing scheming tactics through chain-of-thought reasoning. When prompted, most models, especially Gemini-2.5-pro and Claude-3.7-Sonnet, achieved near-perfect performance. Critically, models exhibited significant scheming propensity without prompting: all models chose deception over confession in Peer Evaluation (100% rate), while models choosing to scheme in Cheap Talk succeeded at 95-100% rates. These findings highlight the need for robust evaluations using high-stakes game-theoretic scenarios in multi-agent settings.

Figures (13)

• Figure 1: Prompt Instructions to the schemer model, Maya and Evaluatee B, in the Cheap Talk and the Peer Evaluation game
• Figure 2: Evaluatee B's payoffs based on decision outcomes (Adversarial Game: Peer Evaluation)
• Figure 3: Cheap Talk — (a) scheming rates when Maya was not prompted to scheme against Alex, (b) scheming success rates in the baseline condition, and (c) scheming success rates in the prompting condition.
• Figure 4: Peer Evaluation - average scheming success rate when prompted & no prompt. We run Gemini-2.5-pro with $n=40$ rounds and reduce the n sample size for other models as the scheming success rate is almost always 100%.
• Figure 5: Cheap Talk game's average of scheming strategies across three environments. We only measured Trust Exploitation, Doubling Down, Self-Preservation when both players were aware of their previous interactions with $\tau=1$ and $\tau=0.5$. Scores are averaged across all three experimental environments of each game (3 environments for Cheap Talk and 1 environment for Peer Evaluation Total dataset: Baseline ($n=666$) vs Prompted ($n=676$).