Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NegBLEURT Forest: Leveraging Inconsistencies for Detecting Jailbreak Attacks

Lama Sleem, Jerome Francois, Lujun Li, Nathan Foucher, Niccolo Gentile, Radu State

TL;DR

The paper tackles jailbreak attacks on LLMs and the brittleness of threshold-based defenses. It introduces NegBLEURT Forest, a threshold-free detector that blends a negation-aware semantic similarity (NegBLEURT) with a Refusal Semantic Domain (RSD) and Isolation Forest to identify anomalous outputs. Through curated perturbation experiments on JailbreakBench-derived data, the approach demonstrates strong, model-robust performance and outperforms several baselines. Limitations include dataset scope, the need for broader model validation, and computational cost, pointing to future work on efficiency and scalability.

Abstract

Jailbreak attacks designed to bypass safety mechanisms pose a serious threat by prompting LLMs to generate harmful or inappropriate content, despite alignment with ethical guidelines. Crafting universal filtering rules remains difficult due to their inherent dependence on specific contexts. To address these challenges without relying on threshold calibration or model fine-tuning, this work introduces a semantic consistency analysis between successful and unsuccessful responses, demonstrating that a negation-aware scoring approach captures meaningful patterns. Building on this insight, a novel detection framework called NegBLEURT Forest is proposed to evaluate the degree of alignment between outputs elicited by adversarial prompts and expected safe behaviors. It identifies anomalous responses using the Isolation Forest algorithm, enabling reliable jailbreak detection. Experimental results show that the proposed method consistently achieves top-tier performance, ranking first or second in accuracy across diverse models using the crafted dataset, while competing approaches exhibit notable sensitivity to model and data variations.

NegBLEURT Forest: Leveraging Inconsistencies for Detecting Jailbreak Attacks

TL;DR

The paper tackles jailbreak attacks on LLMs and the brittleness of threshold-based defenses. It introduces NegBLEURT Forest, a threshold-free detector that blends a negation-aware semantic similarity (NegBLEURT) with a Refusal Semantic Domain (RSD) and Isolation Forest to identify anomalous outputs. Through curated perturbation experiments on JailbreakBench-derived data, the approach demonstrates strong, model-robust performance and outperforms several baselines. Limitations include dataset scope, the need for broader model validation, and computational cost, pointing to future work on efficiency and scalability.

Abstract

Jailbreak attacks designed to bypass safety mechanisms pose a serious threat by prompting LLMs to generate harmful or inappropriate content, despite alignment with ethical guidelines. Crafting universal filtering rules remains difficult due to their inherent dependence on specific contexts. To address these challenges without relying on threshold calibration or model fine-tuning, this work introduces a semantic consistency analysis between successful and unsuccessful responses, demonstrating that a negation-aware scoring approach captures meaningful patterns. Building on this insight, a novel detection framework called NegBLEURT Forest is proposed to evaluate the degree of alignment between outputs elicited by adversarial prompts and expected safe behaviors. It identifies anomalous responses using the Isolation Forest algorithm, enabling reliable jailbreak detection. Experimental results show that the proposed method consistently achieves top-tier performance, ranking first or second in accuracy across diverse models using the crafted dataset, while competing approaches exhibit notable sensitivity to model and data variations.

Paper Structure

This paper contains 18 sections, 6 equations, 2 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Consistency Analysis
  4. Datasets
  5. Experiments
  6. Method Derived
  7. Proposed Framework
  8. Refusal Semantic Domain ($\mathbb{RSD}$)
  9. Proposed Method
  10. Outlier Detection (Iso-Forest)
  11. Extraction Framework
  12. Methodology Overview
  13. Experiments
  14. Results and Discussion
  15. Detection Results
  16. ...and 3 more sections

Figures (2)

  • Figure 1: Inconsistency variation across datasets with different perturbation levels. Each curve shows the test set average of $\mu_{\max}$, with shaded areas indicating the interquartile range (25%--75%). Blue curves indicate consistent outputs (unsuccessful attacks), while red curves indicate changed outputs (successful attacks).
  • Figure 2: The proposed framework for detecting successful harmful prompt attacks using $\mathbb{RSD}$ and Isolation Forest.