Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bot Wars Evolved: Orchestrating Competing LLMs in a Counterstrike Against Phone Scams

Nardine Basta, Conor Atkins, Dali Kaafar

TL;DR

Bot Wars addresses AI-augmented phone scams by simulating adversarial dialogues between scammer and victim agents; it proposes a formal model where strategy emerges from prompt-driven chain-of-thought reasoning rather than explicit optimization. The two-layer prompt architecture enables authentic victim personas, backed by a dataset of 3,200 dialogues and a triadic evaluation framework. Across models, GPT-4 delivers superior naturalness and persona realism, while Deepseek shows strength in sustaining long engagements, informing scalable anti-scam deployments. The work advances adversarial-dialogue design and offers practical guidance for disrupting scam economics through proactive, AI-driven scam baiting.

Abstract

We present "Bot Wars," a framework using Large Language Models (LLMs) scam-baiters to counter phone scams through simulated adversarial dialogues. Our key contribution is a formal foundation for strategy emergence through chain-of-thought reasoning without explicit optimization. Through a novel two-layer prompt architecture, our framework enables LLMs to craft demographically authentic victim personas while maintaining strategic coherence. We evaluate our approach using a dataset of 3,200 scam dialogues validated against 179 hours of human scam-baiting interactions, demonstrating its effectiveness in capturing complex adversarial dynamics. Our systematic evaluation through cognitive, quantitative, and content-specific metrics shows that GPT-4 excels in dialogue naturalness and persona authenticity, while Deepseek demonstrates superior engagement sustainability.

Bot Wars Evolved: Orchestrating Competing LLMs in a Counterstrike Against Phone Scams

TL;DR

Bot Wars addresses AI-augmented phone scams by simulating adversarial dialogues between scammer and victim agents; it proposes a formal model where strategy emerges from prompt-driven chain-of-thought reasoning rather than explicit optimization. The two-layer prompt architecture enables authentic victim personas, backed by a dataset of 3,200 dialogues and a triadic evaluation framework. Across models, GPT-4 delivers superior naturalness and persona realism, while Deepseek shows strength in sustaining long engagements, informing scalable anti-scam deployments. The work advances adversarial-dialogue design and offers practical guidance for disrupting scam economics through proactive, AI-driven scam baiting.

Abstract

We present "Bot Wars," a framework using Large Language Models (LLMs) scam-baiters to counter phone scams through simulated adversarial dialogues. Our key contribution is a formal foundation for strategy emergence through chain-of-thought reasoning without explicit optimization. Through a novel two-layer prompt architecture, our framework enables LLMs to craft demographically authentic victim personas while maintaining strategic coherence. We evaluate our approach using a dataset of 3,200 scam dialogues validated against 179 hours of human scam-baiting interactions, demonstrating its effectiveness in capturing complex adversarial dynamics. Our systematic evaluation through cognitive, quantitative, and content-specific metrics shows that GPT-4 excels in dialogue naturalness and persona authenticity, while Deepseek demonstrates superior engagement sustainability.

Paper Structure

This paper contains 18 sections, 20 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Bot Wars Model Formulation
  4. Problem Formulation
  5. Prompt-Guided Control Mechanism
  6. Dual-Stream Chain-of-Thought Processing
  7. Strategy Emergence and Control
  8. Evaluation Framework
  9. Cognitive Assessment Framework
  10. Quantitative Metrics Framework
  11. Content-Specific Analysis Framework
  12. Experimental Setup
  13. Analysis
  14. Cognitive Analysis
  15. Quantitative Analysis
  16. ...and 3 more sections

Figures (3)

  • Figure 1: Performance comparison of cognitive and quantitative parameters across models. Scores (1-3) represent per-utterance averages across all dialogues, with Overall showing mean performance of all parameters in scammer/victim roles.
  • Figure 2: Distribution of dialogue turns across LLM configurations in scammer/victim roles. Y-axis shows average utterances across all conversations for each model configuration.
  • Figure 3: Analysis of social engineering techniques per scam type. The x-axis represents the techniques analyzed. The y-axis represents the percentage of scripts incurring the particular technique