Improved Large Language Model Jailbreak Detection via Pretrained Embeddings

TL;DR

This work tackles jailbreak detection in enterprise LLM deployments by pairing retrieval-oriented embeddings with conventional machine learning classifiers to identify jailbreak prompts. It evaluates four diverse embedding models and four detector architectures on a large, deduplicated aggregation of jailbreak datasets, demonstrating strong cross-validation performance and superior results on public benchmarks like JailbreakHub. The approach substantially outperforms public LLM-firewall baselines, especially in realistic settings, suggesting a practical, scalable guardrail that can be retrained affordably to adapt to model drift. Overall, the study provides a compelling, deployable direction for improving LLM safety through embedding-based detection coupled with lightweight classifiers.

Abstract

The adoption of large language models (LLMs) in many applications, from customer service chat bots and software development assistants to more capable agentic systems necessitates research into how to secure these systems. Attacks like prompt injection and jailbreaking attempt to elicit responses and actions from these models that are not compliant with the safety, privacy, or content policies of organizations using the model in their application. In order to counter abuse of LLMs for generating potentially harmful replies or taking undesirable actions, LLM owners must apply safeguards during training and integrate additional tools to block the LLM from generating text that abuses the model. Jailbreaking prompts play a vital role in convincing an LLM to generate potentially harmful content, making it important to identify jailbreaking attempts to block any further steps. In this work, we propose a novel approach to detect jailbreak prompts based on pairing text embeddings well-suited for retrieval with traditional machine learning classification algorithms. Our approach outperforms all publicly available methods from open source LLM security applications.