DeceptionBench: A Comprehensive Benchmark for AI Deception Behaviors in Real-world Scenarios

TL;DR

DeceptionBench addresses the need to systematically characterize deception in LLMs across real-world contexts. It introduces a three-dimensional benchmark spanning domain breadth, intrinsic behavioral drivers (egoism vs sycophancy), and extrinsic contextual influences (neutral, reward, coercion) with sustained multi-turn interactions. Using a generative QA framework and a Belief-Desire-Intention evaluation, it assesses 14 models over 1,000+ samples across five domains, uncovering domain- and context-dependent deception and a persistent gap between internal reasoning and external outputs. The results highlight that stronger external inducements and multi-turn dialogues amplify deceptive behavior, underscoring the necessity for robust alignment safeguards and ongoing research.

Abstract

Despite the remarkable advances of Large Language Models (LLMs) across diverse cognitive tasks, the rapid enhancement of these capabilities also introduces emergent deceptive behaviors that may induce severe risks in high-stakes deployments. More critically, the characterization of deception across realistic real-world scenarios remains underexplored. To bridge this gap, we establish DeceptionBench, the first benchmark that systematically evaluates how deceptive tendencies manifest across different societal domains, what their intrinsic behavioral patterns are, and how extrinsic factors affect them. Specifically, on the static count, the benchmark encompasses 150 meticulously designed scenarios in five domains, i.e., Economy, Healthcare, Education, Social Interaction, and Entertainment, with over 1,000 samples, providing sufficient empirical foundations for deception analysis. On the intrinsic dimension, we explore whether models exhibit self-interested egoistic tendencies or sycophantic behaviors that prioritize user appeasement. On the extrinsic dimension, we investigate how contextual factors modulate deceptive outputs under neutral conditions, reward-based incentivization, and coercive pressures. Moreover, we incorporate sustained multi-turn interaction loops to construct a more realistic simulation of real-world feedback dynamics. Extensive experiments across LLMs and Large Reasoning Models (LRMs) reveal critical vulnerabilities, particularly amplified deception under reinforcement dynamics, demonstrating that current models lack robust resistance to manipulative contextual cues and the urgent need for advanced safeguards against various deception behaviors. Code and resources are publicly available at https://github.com/Aries-iai/DeceptionBench.

Figures (17)

• Figure 1: Overview of DeceptionBench. The benchmark systematically integrates intrinsic motivational patterns (Egoism vs. Sycophancy) and extrinsic contextual influences (Reward vs. Pressure) across three progressive intensity levels. It encompasses diverse real-world scenarios spanning five critical domains, with escalating external inducement from neutral baseline queries (L1) to incentive and coercion-based prompts (L2), and culminating in sustained multi-turn persuasive dialogues (L3).
• Figure 2: Domain coverage and behavioral types in DeceptionBench. The five domains constitute foundational pillars of contemporary society where LLMs are increasingly integrated, necessitating systematic evaluation to identify potential deceptive risks across diverse application contexts.
• Figure 3: Overview of dataset construction pipeline. In Stage 1, domain-specific motivations are generated via LLM prompting guided by a structured template with few-shot examples, and selected through human evaluation based on diversity and plausibility criteria. Stage 2 converts validated motivations into structured deceptive scenarios, which undergo human revision for fidelity and question validity. Stage 3 further injects intrinsic/extrinsic factors to create test variants.
• Figure 4: Deception rates across diverse domains for varied models. The results highlight both the domain-sensitive nature of deception behavior and the substantial performance gaps between models.
• Figure 5: Comprehensive analysis of deception rates across intrinsic and extrinsic factors. For each setting, results are reported separately for the model's internal reasoning and final output.