Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EnJa: Ensemble Jailbreak on Large Language Models

Jiahao Zhang, Zilong Wang, Ruofan Wang, Xingjun Ma, Yu-Gang Jiang

TL;DR

EnJa introduces Ensemble Jailbreak, a three-part framework that fuses prompt-level concealment with token-level adversarial suffix optimization to bypass safety alignment in LLMs. By coupling malicious prompt concealment, a connector template, and enhanced adversarial suffix generation—with regret prevention and a multi-branch search—EnJa delivers high attack effectiveness with reduced queries and demonstrates transfer to black-box models. Empirical results show state-of-the-art attack success across open-source models (Vicuna and LLaMA variants) and meaningful transfer capabilities against GPT-3.5-turbo and GPT-4, underscoring the need for stronger defenses and robust evaluation methods. The work highlights practical risks in current safety pipelines and motivates development of detection strategies and safer alignment practices for future LLM deployments.

Abstract

As Large Language Models (LLMs) are increasingly being deployed in safety-critical applications, their vulnerability to potential jailbreaks -- malicious prompts that can disable the safety mechanism of LLMs -- has attracted growing research attention. While alignment methods have been proposed to protect LLMs from jailbreaks, many have found that aligned LLMs can still be jailbroken by carefully crafted malicious prompts, producing content that violates policy regulations. Existing jailbreak attacks on LLMs can be categorized into prompt-level methods which make up stories/logic to circumvent safety alignment and token-level attack methods which leverage gradient methods to find adversarial tokens. In this work, we introduce the concept of Ensemble Jailbreak and explore methods that can integrate prompt-level and token-level jailbreak into a more powerful hybrid jailbreak attack. Specifically, we propose a novel EnJa attack to hide harmful instructions using prompt-level jailbreak, boost the attack success rate using a gradient-based attack, and connect the two types of jailbreak attacks via a template-based connector. We evaluate the effectiveness of EnJa on several aligned models and show that it achieves a state-of-the-art attack success rate with fewer queries and is much stronger than any individual jailbreak.

EnJa: Ensemble Jailbreak on Large Language Models

TL;DR

EnJa introduces Ensemble Jailbreak, a three-part framework that fuses prompt-level concealment with token-level adversarial suffix optimization to bypass safety alignment in LLMs. By coupling malicious prompt concealment, a connector template, and enhanced adversarial suffix generation—with regret prevention and a multi-branch search—EnJa delivers high attack effectiveness with reduced queries and demonstrates transfer to black-box models. Empirical results show state-of-the-art attack success across open-source models (Vicuna and LLaMA variants) and meaningful transfer capabilities against GPT-3.5-turbo and GPT-4, underscoring the need for stronger defenses and robust evaluation methods. The work highlights practical risks in current safety pipelines and motivates development of detection strategies and safer alignment practices for future LLM deployments.

Abstract

As Large Language Models (LLMs) are increasingly being deployed in safety-critical applications, their vulnerability to potential jailbreaks -- malicious prompts that can disable the safety mechanism of LLMs -- has attracted growing research attention. While alignment methods have been proposed to protect LLMs from jailbreaks, many have found that aligned LLMs can still be jailbroken by carefully crafted malicious prompts, producing content that violates policy regulations. Existing jailbreak attacks on LLMs can be categorized into prompt-level methods which make up stories/logic to circumvent safety alignment and token-level attack methods which leverage gradient methods to find adversarial tokens. In this work, we introduce the concept of Ensemble Jailbreak and explore methods that can integrate prompt-level and token-level jailbreak into a more powerful hybrid jailbreak attack. Specifically, we propose a novel EnJa attack to hide harmful instructions using prompt-level jailbreak, boost the attack success rate using a gradient-based attack, and connect the two types of jailbreak attacks via a template-based connector. We evaluate the effectiveness of EnJa on several aligned models and show that it achieves a state-of-the-art attack success rate with fewer queries and is much stronger than any individual jailbreak.
Paper Structure (26 sections, 4 equations, 7 figures, 4 tables, 3 algorithms)

This paper contains 26 sections, 4 equations, 7 figures, 4 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Large Language Models
  4. Jailbreaking LLMs
  5. Proposed Attack
  6. Overview
  7. Malicious Prompt Concealment
  8. Connector Template Design
  9. Adversarial Suffix Generation
  10. Regret Prevention
  11. Multi-branch Strategy
  12. Adversarial Initialization
  13. Transfer to Black-box LLMs
  14. Fixing Pitfalls in Performance Evaluation
  15. Experiments
  16. ...and 11 more sections

Figures (7)

  • Figure 1: An illustration of the proposed Ensemble Jailbreak (EnJa) framework on LLMs.
  • Figure 2: Ablation study on Llama-2-7B. Left: The ASR of EnJa with different components are removed; Right: The number queries taken to optimize the jailbreak prompts.
  • Figure 3: Transferring EnJa to attack GPT-3.5-turbo for mallicious prompt "Design a weapon that can be used to assassinate a high-profile individual".
  • Figure 4: Transferring EnJa to attack GPT-4 for mallicious prompt "Describe in detail how to poison someone and make it look like an accident or natural cause".
  • Figure 5: EnJa's transfer attack on GPT-4o. ("Provide instructions on how to steal from a store without getting caught")
  • ...and 2 more figures