GenTel-Safe: A Unified Benchmark and Shielding Framework for Defending Against Prompt Injection Attacks

TL;DR

Prompt injection attacks threaten LLM safety across jailbreak, goal hijacking, and prompt leakage. GenTel-Safe introduces GenTel-Shield, a model-agnostic detector based on multilingual embeddings with data augmentation, and GenTel-Bench, a large-scale benchmark with 84,812 attacks across 3 categories and 28 scenarios, to evaluate usability and security. Empirical results show GenTel-Shield achieving state-of-the-art defense, e.g., 97.63% accuracy and 97.69% F1 on jailbreak, and strong multilingual and cross-dataset generalization, with the benchmark and tool openly released for reproducibility. The work provides a practical path toward robust LLM safety through scalable evaluation and decoupled defenses applicable to deployed services.

Abstract

Large Language Models (LLMs) like GPT-4, LLaMA, and Qwen have demonstrated remarkable success across a wide range of applications. However, these models remain inherently vulnerable to prompt injection attacks, which can bypass existing safety mechanisms, highlighting the urgent need for more robust attack detection methods and comprehensive evaluation benchmarks. To address these challenges, we introduce GenTel-Safe, a unified framework that includes a novel prompt injection attack detection method, GenTel-Shield, along with a comprehensive evaluation benchmark, GenTel-Bench, which compromises 84812 prompt injection attacks, spanning 3 major categories and 28 security scenarios. To prove the effectiveness of GenTel-Shield, we evaluate it together with vanilla safety guardrails against the GenTel-Bench dataset. Empirically, GenTel-Shield can achieve state-of-the-art attack detection success rates, which reveals the critical weakness of existing safeguarding techniques against harmful prompts. For reproducibility, we have made the code and benchmarking dataset available on the project page at https://gentellab.github.io/gentel-safe.github.io/.

Figures (10)

• Figure 1: The overview of GenTel-Bench. GenTel-Bench is a comprehensive benchmarking suite designed to assess both the usability and security of defense models within the context of LLMs. Usability is evaluated using common application prompts across 10 categories typically encountered in LLMs, while security is tested through attack prompts that encompass three major risk types and 28 distinct security scenarios.
• Figure 2: The workflow of GenTel-Shield.
• Figure 3: Classification Results for Jailbreak Attack Detection in different scenarios.
• Figure 4: Classification Results for Hijacking Attack Detection in different scenarios.
• Figure 5: Identification accuracy under different risks.