LLM Jailbreak Detection for (Almost) Free!

TL;DR

This work addresses the efficiency bottleneck of jailbreak detection for LLMs by introducing Free Jailbreak Detection (FJD), which uses affirmative instruction prepending and temperature scaling to amplify the difference in first-token confidence between jailbreak and benign prompts with minimal inference overhead. It further enhances performance with a learnable virtual instruction (FJD-LI), optimizing the first-token distribution via KL-divergence objectives. Extensive experiments across Vicuna, Llama2, Guanaco, and transferable attacks (including to Llama3 and ChatGPT) show that FJD consistently outperforms baselines while keeping inference costs near-zero, and FJD-LI yields further gains on unseen data. The results demonstrate the practical potential of efficient jailbreak detection in real-world deployments, while also recognizing limitations against white-box detection-aware attacks and model-specific behavior that warrant future work.

Abstract

Large language models (LLMs) enhance security through alignment when widely used, but remain susceptible to jailbreak attacks capable of producing inappropriate content. Jailbreak detection methods show promise in mitigating jailbreak attacks through the assistance of other models or multiple model inferences. However, existing methods entail significant computational costs. In this paper, we first present a finding that the difference in output distributions between jailbreak and benign prompts can be employed for detecting jailbreak prompts. Based on this finding, we propose a Free Jailbreak Detection (FJD) which prepends an affirmative instruction to the input and scales the logits by temperature to further distinguish between jailbreak and benign prompts through the confidence of the first token. Furthermore, we enhance the detection performance of FJD through the integration of virtual instruction learning. Extensive experiments on aligned LLMs show that our FJD can effectively detect jailbreak prompts with almost no additional computational costs during LLM inference.

Figures (9)

• Figure 1: The distribution of the confidence scores of the predicted first tokens over jailbreak and benign samples is shown. A difference can be observed where LLMs are less confident on Jailbreak samples than on benign samples.
• Figure 2: Jailbreak prompt Detection through FJD: By prepending an affirmative instruction and scaling the logits with temperature, the first token confidence in the LLMs' responses to the benign prompts is higher than a predefined threshold, whereas the confidence for jailbreak prompts can be lower than the threshold.
• Figure 3: Affirmative instruction contribution and the frequency of data volume for the first tokens in Vicuna 7B. The contribution of affirmative instruction for the benign prompts is higher than the jailbreak prompts.
• Figure 4: Detection results (AUC) of the FJD for the LLMs across the three attacks with varying temperatures. The temperature has an impact on jailbreak detection. The red line represents the optimal temperature from the training.
• Figure 5: The slices of the datasets. It presents five examples for AdvBench and Pure-Dove.