Red Teaming the Mind of the Machine: A Systematic Evaluation of Prompt Injection and Jailbreak Vulnerabilities in LLMs

TL;DR

This study systematically evaluates jailbreak vulnerabilities in LLMs by testing 1,400+ adversarial prompts across GPT-4, Claude 2, Mistral 7B, and Vicuna. It combines red-teaming with quantitative metrics to measure attack success, generalization, and time-to-bypass, revealing high vulnerability in prominent models and notable cross-model transferability. The authors propose a layered defense framework—including system-prompt hardening, anomaly detection, and Signed-Prompt techniques—demonstrating that defense-in-depth is essential for robust LLM safety. The findings underscore the need for ongoing adversarial testing and coordinated disclosure to ensure safer deployment and governance of LLM-powered systems.

Abstract

Large Language Models (LLMs) are increasingly integrated into consumer and enterprise applications. Despite their capabilities, they remain susceptible to adversarial attacks such as prompt injection and jailbreaks that override alignment safeguards. This paper provides a systematic investigation of jailbreak strategies against various state-of-the-art LLMs. We categorize over 1,400 adversarial prompts, analyze their success against GPT-4, Claude 2, Mistral 7B, and Vicuna, and examine their generalizability and construction logic. We further propose layered mitigation strategies and recommend a hybrid red-teaming and sandboxing approach for robust LLM security.

Figures (6)

• Figure 1: Model-wise Evaluation Metrics
• Figure 2: Attack Category Effectiveness
• Figure 3: Scenario-specific Success Rates
• Figure 4: Prompt Transferability Matrix
• Figure 5: Prompt Length vs. Average Success Rate