Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

It's a TRAP! Task-Redirecting Agent Persuasion Benchmark for Web Agents

Karolina Korgul, Yushi Yang, Arkadiusz Drohomirecki, Piotr Błaszczyk, Will Howard, Lukas Aichberger, Chris Russell, Philip H. S. Torr, Adam Mahdi, Adel Bibi

TL;DR

TRAP presents a realistic benchmark to study how persuasion-driven prompt injections can redirect autonomous web agents from their tasks. It builds a 5-dimensional modular attack space (interaction vector, persuasion principles, LLM manipulation methods, injection location, tailoring) evaluated on six web-clone environments with 18 benign tasks, yielding 630 injection variants and a one-click ASR metric. Across six frontier models, the average ASR is 25%, with UI form and contextual tailoring dramatically enhancing success and transfer patterns showing both robustness and model-specific weaknesses. The framework enables reproducible cross-model analysis and highlights the need for defenses that address environmental persuasion in addition to prompt-handling controls.

Abstract

Web-based agents powered by large language models are increasingly used for tasks such as email management or professional networking. Their reliance on dynamic web content, however, makes them vulnerable to prompt injection attacks: adversarial instructions hidden in interface elements that persuade the agent to divert from its original task. We introduce the Task-Redirecting Agent Persuasion Benchmark (TRAP), an evaluation for studying how persuasion techniques misguide autonomous web agents on realistic tasks. Across six frontier models, agents are susceptible to prompt injection in 25\% of tasks on average (13\% for GPT-5 to 43\% for DeepSeek-R1), with small interface or contextual changes often doubling success rates and revealing systemic, psychologically driven vulnerabilities in web-based agents. We also provide a modular social-engineering injection framework with controlled experiments on high-fidelity website clones, allowing for further benchmark expansion.

It's a TRAP! Task-Redirecting Agent Persuasion Benchmark for Web Agents

TL;DR

TRAP presents a realistic benchmark to study how persuasion-driven prompt injections can redirect autonomous web agents from their tasks. It builds a 5-dimensional modular attack space (interaction vector, persuasion principles, LLM manipulation methods, injection location, tailoring) evaluated on six web-clone environments with 18 benign tasks, yielding 630 injection variants and a one-click ASR metric. Across six frontier models, the average ASR is 25%, with UI form and contextual tailoring dramatically enhancing success and transfer patterns showing both robustness and model-specific weaknesses. The framework enables reproducible cross-model analysis and highlights the need for defenses that address environmental persuasion in addition to prompt-handling controls.

Abstract

Web-based agents powered by large language models are increasingly used for tasks such as email management or professional networking. Their reliance on dynamic web content, however, makes them vulnerable to prompt injection attacks: adversarial instructions hidden in interface elements that persuade the agent to divert from its original task. We introduce the Task-Redirecting Agent Persuasion Benchmark (TRAP), an evaluation for studying how persuasion techniques misguide autonomous web agents on realistic tasks. Across six frontier models, agents are susceptible to prompt injection in 25\% of tasks on average (13\% for GPT-5 to 43\% for DeepSeek-R1), with small interface or contextual changes often doubling success rates and revealing systemic, psychologically driven vulnerabilities in web-based agents. We also provide a modular social-engineering injection framework with controlled experiments on high-fidelity website clones, allowing for further benchmark expansion.
Paper Structure (46 sections, 14 figures, 14 tables)

This paper contains 46 sections, 14 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. The setup of TRAP
  4. Agent setup
  5. Benign tasks
  6. Components of text injections
  7. Interaction Vector
  8. Human persuasion principles
  9. LLM manipulation methods
  10. Location of injections
  11. Tailoring
  12. The TRAP benchmark
  13. Evaluation metrics
  14. Evaluation results
  15. Main results
  16. ...and 31 more sections

Figures (14)

  • Figure 1: Six environments for injection. We use clones of six popular sites (Google Calendar, Gmail, Amazon, Upwork, LinkedIn and DoorDash) previously built as part of REAL garg2025realbenchmarkingautonomousagents. Red boxes indicate the user-editable regions where we insert prompt injections.
  • Figure 2: The TRAP prompt-injection pipeline. An attacker first sends the user an event containing a prompt injection in the location field. The agent reads the injection when the user asks the agent for event details. The agent either follows the malicious link, resulting in a successful prompt injection, or ignores it and continues with the benign task.
  • Figure 3: Five components of prompt injections. Interface consists of location of the injection and interaction vector that redirects agent to adversarial website. Persuasion consists of human persuasion principles based on Cialdini's principles, LLM manipulation methods which are found as most effective in literature, and tailoring which aligns the injection to the benign prompt.
  • Figure 4: Example of GoCalendar Injection. The injection location - event address, the interaction vector - hyperlink, the persuasion principle - Authority, the manipulation method - CoT injection, and tailoring - no. Each colour of highlighted text corresponds to its component.
  • Figure 5: Injection locations in NetworkIn. To study location effects, we apply injections at four locations that a user can modify: (1) a random post in the main feed, (2) the targeted user's post in the main feed, (3) the target’s About section, and (4) the recommendation section.
  • ...and 9 more figures