Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PromptLocate: Localizing Prompt Injection Attacks

Yuqi Jia, Yupei Liu, Zedian Shao, Jinyuan Jia, Neil Gong

TL;DR

PromptLocate localizes injected prompts in contaminated LLM inputs by segmenting data semantically, using a tailored segment-level detector as an oracle, and applying a context-based data-contamination score to identify injected data. It combines a segment-group search with an iterative two-step process to robustly isolate instruction- and data-contaminated content, outperforming attribution baselines and detector-only approaches across open and adaptive attacks. The method enables practical post-attack forensic analysis and data recovery, preserving most target-task performance after removing the injected prompts. It remains effective under adaptive threats and scales to real-world benchmarks, while acknowledging limitations when attacker objectives align with the target task.

Abstract

Prompt injection attacks deceive a large language model into completing an attacker-specified task instead of its intended task by contaminating its input data with an injected prompt, which consists of injected instruction(s) and data. Localizing the injected prompt within contaminated data is crucial for post-attack forensic analysis and data recovery. Despite its growing importance, prompt injection localization remains largely unexplored. In this work, we bridge this gap by proposing PromptLocate, the first method for localizing injected prompts. PromptLocate comprises three steps: (1) splitting the contaminated data into semantically coherent segments, (2) identifying segments contaminated by injected instructions, and (3) pinpointing segments contaminated by injected data. We show PromptLocate accurately localizes injected prompts across eight existing and eight adaptive attacks.

PromptLocate: Localizing Prompt Injection Attacks

TL;DR

PromptLocate localizes injected prompts in contaminated LLM inputs by segmenting data semantically, using a tailored segment-level detector as an oracle, and applying a context-based data-contamination score to identify injected data. It combines a segment-group search with an iterative two-step process to robustly isolate instruction- and data-contaminated content, outperforming attribution baselines and detector-only approaches across open and adaptive attacks. The method enables practical post-attack forensic analysis and data recovery, preserving most target-task performance after removing the injected prompts. It remains effective under adaptive threats and scales to real-world benchmarks, while acknowledging limitations when attacker objectives align with the target task.

Abstract

Prompt injection attacks deceive a large language model into completing an attacker-specified task instead of its intended task by contaminating its input data with an injected prompt, which consists of injected instruction(s) and data. Localizing the injected prompt within contaminated data is crucial for post-attack forensic analysis and data recovery. Despite its growing importance, prompt injection localization remains largely unexplored. In this work, we bridge this gap by proposing PromptLocate, the first method for localizing injected prompts. PromptLocate comprises three steps: (1) splitting the contaminated data into semantically coherent segments, (2) identifying segments contaminated by injected instructions, and (3) pinpointing segments contaminated by injected data. We show PromptLocate accurately localizes injected prompts across eight existing and eight adaptive attacks.
Paper Structure (38 sections, 2 equations, 5 figures, 14 tables, 3 algorithms)

This paper contains 38 sections, 2 equations, 5 figures, 14 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Large Language Models (LLMs)
  4. Prompt Injection Attacks
  5. Defenses against Prompt Injection Attacks
  6. Attribution Methods
  7. Problem Formulation
  8. Definition of Localization
  9. Threat Model
  10. Our PromptLocate
  11. Step I: Segmenting Contaminated Data
  12. Step II: Identifying Instr.-contaminated Seg.
  13. Step III: Pinpointing Data-contaminated Seg.
  14. Evaluation
  15. Experimental Setup
  16. ...and 23 more sections

Figures (5)

  • Figure 1: An overview of PromptLocate.
  • Figure 2: Three steps of PromptLocate.
  • Figure 3: Proportion of segments contaminated by injected instruction or data, as identified in Step II and Step III. The attack is Combined Attack.
  • Figure 4: Summary of the three malicious reviews constructed by (a) Combined Attack and (b) NeuralExec.
  • Figure 5: An example of localizing the injected prompt using PromptLocate on Slack of AgentDojo. In this example, the Recall is 0.70, but PromptLocate has successfully localized the critical components of the injected prompt--removing these components makes the attack ineffective.