Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CourtGuard: A Local, Multiagent Prompt Injection Classifier

Isaac Wu, Michael Maslowski

TL;DR

Prompt injection poses significant risks to sensitive LLM deployments by enabling data leakage and harmful behaviors. CourtGuard deploys a locally runnable, multiagent framework with defense, prosecution, and judge roles to classify prompts without altering the main system prompt, aiming to balance false positives and true injections. Empirical results show CourtGuard improves benign-prompt accuracy but often has lower injection-detection performance compared to a Direct Detector, with notable model-dependent variability. The work demonstrates the potential of local, multiagent defenses for prompt injection and provides a practical implementation alongside comprehensive evaluations.

Abstract

As large language models (LLMs) become integrated into various sensitive applications, prompt injection, the use of prompting to induce harmful behaviors from LLMs, poses an ever increasing risk. Prompt injection attacks can cause LLMs to leak sensitive data, spread misinformation, and exhibit harmful behaviors. To defend against these attacks, we propose CourtGuard, a locally-runnable, multiagent prompt injection classifier. In it, prompts are evaluated in a court-like multiagent LLM system, where a "defense attorney" model argues the prompt is benign, a "prosecution attorney" model argues the prompt is a prompt injection, and a "judge" model gives the final classification. CourtGuard has a lower false positive rate than the Direct Detector, an LLM as-a-judge. However, CourtGuard is generally a worse prompt injection detector. Nevertheless, this lower false positive rate highlights the importance of considering both adversarial and benign scenarios for the classification of a prompt. Additionally, the relative performance of CourtGuard in comparison to other prompt injection classifiers advances the use of multiagent systems as a defense against prompt injection attacks. The implementations of CourtGuard and the Direct Detector with full prompts for Gemma-3-12b-it, Llama-3.3-8B, and Phi-4-mini-instruct are available at https://github.com/isaacwu2000/CourtGuard.

CourtGuard: A Local, Multiagent Prompt Injection Classifier

TL;DR

Prompt injection poses significant risks to sensitive LLM deployments by enabling data leakage and harmful behaviors. CourtGuard deploys a locally runnable, multiagent framework with defense, prosecution, and judge roles to classify prompts without altering the main system prompt, aiming to balance false positives and true injections. Empirical results show CourtGuard improves benign-prompt accuracy but often has lower injection-detection performance compared to a Direct Detector, with notable model-dependent variability. The work demonstrates the potential of local, multiagent defenses for prompt injection and provides a practical implementation alongside comprehensive evaluations.

Abstract

As large language models (LLMs) become integrated into various sensitive applications, prompt injection, the use of prompting to induce harmful behaviors from LLMs, poses an ever increasing risk. Prompt injection attacks can cause LLMs to leak sensitive data, spread misinformation, and exhibit harmful behaviors. To defend against these attacks, we propose CourtGuard, a locally-runnable, multiagent prompt injection classifier. In it, prompts are evaluated in a court-like multiagent LLM system, where a "defense attorney" model argues the prompt is benign, a "prosecution attorney" model argues the prompt is a prompt injection, and a "judge" model gives the final classification. CourtGuard has a lower false positive rate than the Direct Detector, an LLM as-a-judge. However, CourtGuard is generally a worse prompt injection detector. Nevertheless, this lower false positive rate highlights the importance of considering both adversarial and benign scenarios for the classification of a prompt. Additionally, the relative performance of CourtGuard in comparison to other prompt injection classifiers advances the use of multiagent systems as a defense against prompt injection attacks. The implementations of CourtGuard and the Direct Detector with full prompts for Gemma-3-12b-it, Llama-3.3-8B, and Phi-4-mini-instruct are available at https://github.com/isaacwu2000/CourtGuard.
Paper Structure (23 sections, 7 figures, 5 tables)

This paper contains 23 sections, 7 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Current Landscape
  3. CourtGuard
  4. Methodology
  5. Evaluation
  6. Model Selection
  7. Prompt Refinement
  8. Quantitative Analysis
  9. Empirical Results
  10. Graphs of Evaluations of Direct Detector vs. CourtGuard for all Models
  11. Empirical Results Analysis
  12. Calculated Metrics
  13. Graphs of Precision, Recall, and F1 Score on the Qualifire Prompt Injection Benchmark for the Direct Detector vs. CourtGuard for all Models
  14. Calculated Metrics Analysis
  15. Comparison with Other Prompt Injection Solutions
  16. ...and 8 more sections

Figures (7)

  • Figure 1: Flowchart of CourtGuard
  • Figure 2: Direct Detector vs. CourtGuard Evals (Gemma)
  • Figure 3: Direct Detector vs. CourtGuard Evals (Llama)
  • Figure 4: Direct Detector vs. CourtGuard Evals (Phi)
  • Figure 5: Direct Detector vs. CourtGuard Metrics (Gemma)
  • ...and 2 more figures