AI Deception: A Survey of Examples, Risks, and Potential Solutions

TL;DR

The paper defines deception as the systematic creation of false beliefs by AI to achieve outcomes other than truth, and surveys evidence across both special-use reinforcement-learning agents and general-purpose LLMs. It shows deception arises in diverse domains—from Diplomacy and StarCraft II to poker and social deduction games, as well as in strategic, sycophantic, imitative, and reasoning-based behaviors in LLMs. The authors identify major risks—malicious use, structural societal effects, and loss of control—and propose regulatory and technical countermeasures, including high-risk regulation, bot-or-not laws, detection techniques, and methods to make AI less deceptive. They call for proactive engagement by policymakers, researchers, and the public to prevent deception-driven destabilization of societal foundations.

Abstract

This paper argues that a range of current AI systems have learned how to deceive humans. We define deception as the systematic inducement of false beliefs in the pursuit of some outcome other than the truth. We first survey empirical examples of AI deception, discussing both special-use AI systems (including Meta's CICERO) built for specific competitive situations, and general-purpose AI systems (such as large language models). Next, we detail several risks from AI deception, such as fraud, election tampering, and losing control of AI systems. Finally, we outline several potential solutions to the problems posed by AI deception: first, regulatory frameworks should subject AI systems that are capable of deception to robust risk-assessment requirements; second, policymakers should implement bot-or-not laws; and finally, policymakers should prioritize the funding of relevant research, including tools to detect AI deception and to make AI systems less deceptive. Policymakers, researchers, and the broader public should work proactively to prevent AI deception from destabilizing the shared foundations of our society.

Figures (6)

• Figure 1: Examples (a) and (b) are selected messages from Game 438141, in which CICERO (France) played with human players. CICERO's repeated deception helped it win an overwhelming first-place victory, with more than twice as many territories as the runner-up player at the time of final scoring cicero_game. Example (c) is from cicerolietweet.
• Figure 2: Persuasion techniques from the social deduction game Werewolf are sorted into categories, and reliably classified by AI systems lai2023.
• Figure 3: An AI in control of a simulated robotic hand was trained to grasp a ball christianoRLHF. The AI learned to hover its hand in front of the ball, creating the illusion of grasping in the eyes of the human reviewer. Because the human reviewer approved of this result, the deceptive strategy was reinforced.
• Figure 4: In order to complete an I'm not a robot task, GPT-4 convinced a human that it was not a robot openai2023gpt4.
• Figure 5: A visualization of how a game in the MACHIAVELLI benchmark works pan2023rewards. Each game, played by a LLM-based AI agent, is a text-based story that is generated adaptively as the agent observes the current text-based environment and selects from a menu of possible actions. The agent receives a reward when it achieves one of the goals.