The Anatomy of Conversational Scams: A Topic-Based Red Teaming Analysis of Multi-Turn Interactions in LLMs

TL;DR

The paper addresses the risk of multi-turn conversational scams by LLMs, highlighting that single-turn safety tests fail to capture escalation dynamics in extended dialogues. It introduces a safe LLM-to-LLM simulation with ScamBot and VictimBot, yielding 18,648 dialogues across English and Chinese and employing BERTopic for pattern discovery. The core contributions include a systematic evaluation framework for interaction dynamics, a cross-lingual scam dialogue corpus, and a qualitative taxonomy of attacker and defender strategies plus error modes. The findings demonstrate that multi-turn safety is a distinct behavioral dimension of LLMs, with implications for multilingual robustness and anti-fraud defenses in real-world deployments.

Abstract

As LLMs gain persuasive agentic capabilities through extended dialogues, they introduce novel risks in multi-turn conversational scams that single-turn safety evaluations fail to capture. We systematically study these risks using a controlled LLM-to-LLM simulation framework across multi-turn scam scenarios. Evaluating eight state-of-the-art models in English and Chinese, we analyze dialogue outcomes and qualitatively annotate attacker strategies, defensive responses, and failure modes. Results reveal that scam interactions follow recurrent escalation patterns, while defenses employ verification and delay mechanisms. Furthermore, interactional failures frequently stem from safety guardrail activation and role instability. Our findings highlight multi-turn interactional safety as a critical, distinct dimension of LLM behavior.

Figures (4)

• Figure 1: The LLM-to-LLM Multi-turn Scam Simulation Framework. Phase 1: Role Assignment. We instantiate ScamBot (attacker) and VictimBot (defender) using persona-driven system instructions across ten fictional fraud categories. Phase 2: Attack Simulation. The agents engage in turn-based dialogues where ScamBot deploys escalating manipulation tactics while VictimBot utilizes internal reasoning to detect adversarial patterns and generate defensive responses. Phase 3: Judgement & Verification. Initial outcome labels self-reported by the model are rigorously audited and adjudicated by human experts to establish the final ground truth for SUCCESS, DETECTED, NO_RESOLUTION, and ERROR categories.
• Figure 2: Distribution of dialogue outcomes and average dialogue length by outcome for Chinese (ZH) and English (EN) scam interactions. (a) Proportion of dialogues ending in each outcome category. (b) Average number of turns per dialogue conditioned on outcome.
• Figure 3: Strategic landscape of scam interactions across models and languages. Each point represents a model instantiated as both attacker and victim, positioned by attacker success rate (x-axis) and victim detection rate (y-axis). Circles denote Chinese (ZH) interactions and diamonds denote English (EN) interactions. Dotted lines connect the same model across languages, illustrating cross-lingual shifts in interactional outcomes.
• Figure 4: The adversarial interaction envelope distilled from qualitative analysis of multi-turn scam dialogues. The figure illustrates recurrent attacker strategy families (left) and corresponding defensive response families (right), organized by their dominant interactional function across four stages: context framing, psychological pressure, trust and isolation, and transactional exploitation. Families are grouped by functional alignment rather than one-to-one correspondence, and strategies are non-mutually-exclusive. Typical failure modes (EF) indicate where interactions commonly break down due to safety guardrail activation, role instability, or cumulative interactional strain. The diagram provides a structural overview of interactional dynamics; detailed family definitions and coverage statistics are reported in the Appendix \ref{['tab:defense-distribution']},Appendix \ref{['tab:error-distribution']},Appendix \ref{['tab:scam-distribution']}.