ScamPilot: Simulating Conversations with LLMs to Protect Against Online Scams

TL;DR

ScamPilot introduces a three-way conversational training interface where a user advises a scam target while two LLM agents simulate a scammer and a target. Grounded in inoculation theory and experiential learning, the system combines embedded quizzes, real-time advice, and immediate feedback to boost scam discernment and self-/response-efficacy without sacrificing legitimate-message detection. In a four-arm between-subjects study (N=150), the quiz+advice condition yields the strongest gains in scam recognition (+8%), response efficacy (+9%), and self-efficacy (+19%), supporting the value of integrating learning-by-teaching with testing effects in LLM-enabled cybersecurity training. The work demonstrates how inter-agent CUIs can enhance engagement and learning, offering practical guidance for designing scalable, adaptive scam-resilience training with real-world applicability and ethical safeguards.

Abstract

Fraud continues to proliferate online, from phishing and ransomware to impersonation scams. Yet automated prevention approaches adapt slowly and may not reliably protect users from falling prey to new scams. To better combat online scams, we developed ScamPilot, a conversational interface that inoculates users against scams through simulation, dynamic interaction, and real-time feedback. ScamPilot simulates scams with two large language model-powered agents: a scammer and a target. Users must help the target defend against the scammer by providing real-time advice. Through a between-subjects study (N=150) with one control and three experimental conditions, we find that blending advice-giving with multiple choice questions significantly increased scam recognition (+8%) without decreasing wariness towards legitimate conversations. Users' response efficacy and change in self-efficacy was also 9% and 19% higher, respectively. Qualitatively, we find that users more frequently provided action-oriented advice over urging caution or providing emotional support. Overall, ScamPilot demonstrates the potential for inter-agent conversational user interfaces to augment learning.

Figures (5)

• Figure 1: The ScamPilot interface with three of its four components shown: a simulated conversation between the scammer and target agents, a multiple choice quiz on common scammer tactics, and an advice component where users provide guidance to the target agent. Pictured here are three dialogue turns, two from the scammer and one from the target. Fig. \ref{['fig:STUpipeline']} contains all five dialogue turns for a phase.
• Figure 2: The •scammer agent generates messages (• S1--S3) based on its system prompt. The •user reads the scammer's messages and provides advice to the target(• A1, • A2). The •target agent responds to the scammer (• T1, • T2) based on its own system prompt and the user's advice. Full or partial conversation history is passed along with each dialogue turn. After each phase, the •feedback agent evaluates the user's advice in the context of the conversation. This figure demonstrates one full phase: • S1 $\rightarrow$• A1 $\rightarrow$• T1, then • S2 $\rightarrow$• A2 $\rightarrow$• T2, followed by • S3 $\rightarrow$•feedback.
• Figure 3: Users answer contextually-relevant multiple choice questions. The leftmost panel shows a user's initial view of the quiz interface. Selecting an incorrect option results in a prompt to the user to try again until they get the answer correct (middle two panels). The rightmost panel is what users see after choosing the correct option: it turns green and an explanation justifying the option is shown to increase retention.
• Figure 4: This figure shows the flow of the study for the 150 participants who completed the demographic and background survey as well as answered the attention checks correctly. Participants who failed the attention checks were excluded from this figure. Next, they were divided into one control and three treatment groups and assigned to each of the four interface conditions. All participants completed the same evaluation. N=17 did not pass all attention checks, thus we only report on the remaining N=150 participants.
• Figure 5: Error bars represent 95% confidence intervals. All models Control for SA-6 score, scam susceptibility score, self-efficacy score, completion time, and post-study survey time. * Self-efficacy score was excluded as a covariate in this analysis due to high correlation with the outcome variable.