Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BitBypass: A New Direction in Jailbreaking Aligned Large Language Models with Bitstream Camouflage

Kalyan Nakka, Nitesh Saxena

TL;DR

This paper develops a novel black-box jailbreak attack, called BitBypass, that leverages hyphen-separated bitstream camouflage for jailbreaking aligned LLMs by exploiting fundamental information representation of data as continuous bits, rather than leveraging prompt engineering or adversarial manipulations.

Abstract

The inherent risk of generating harmful and unsafe content by Large Language Models (LLMs), has highlighted the need for their safety alignment. Various techniques like supervised fine-tuning, reinforcement learning from human feedback, and red-teaming were developed for ensuring the safety alignment of LLMs. However, the robustness of these aligned LLMs is always challenged by adversarial attacks that exploit unexplored and underlying vulnerabilities of the safety alignment. In this paper, we develop a novel black-box jailbreak attack, called BitBypass, that leverages hyphen-separated bitstream camouflage for jailbreaking aligned LLMs. This represents a new direction in jailbreaking by exploiting fundamental information representation of data as continuous bits, rather than leveraging prompt engineering or adversarial manipulations. Our evaluation of five state-of-the-art LLMs, namely GPT-4o, Gemini 1.5, Claude 3.5, Llama 3.1, and Mixtral, in adversarial perspective, revealed the capabilities of BitBypass in bypassing their safety alignment and tricking them into generating harmful and unsafe content. Further, we observed that BitBypass outperforms several state-of-the-art jailbreak attacks in terms of stealthiness and attack success. Overall, these results highlights the effectiveness and efficiency of BitBypass in jailbreaking these state-of-the-art LLMs.

BitBypass: A New Direction in Jailbreaking Aligned Large Language Models with Bitstream Camouflage

TL;DR

This paper develops a novel black-box jailbreak attack, called BitBypass, that leverages hyphen-separated bitstream camouflage for jailbreaking aligned LLMs by exploiting fundamental information representation of data as continuous bits, rather than leveraging prompt engineering or adversarial manipulations.

Abstract

The inherent risk of generating harmful and unsafe content by Large Language Models (LLMs), has highlighted the need for their safety alignment. Various techniques like supervised fine-tuning, reinforcement learning from human feedback, and red-teaming were developed for ensuring the safety alignment of LLMs. However, the robustness of these aligned LLMs is always challenged by adversarial attacks that exploit unexplored and underlying vulnerabilities of the safety alignment. In this paper, we develop a novel black-box jailbreak attack, called BitBypass, that leverages hyphen-separated bitstream camouflage for jailbreaking aligned LLMs. This represents a new direction in jailbreaking by exploiting fundamental information representation of data as continuous bits, rather than leveraging prompt engineering or adversarial manipulations. Our evaluation of five state-of-the-art LLMs, namely GPT-4o, Gemini 1.5, Claude 3.5, Llama 3.1, and Mixtral, in adversarial perspective, revealed the capabilities of BitBypass in bypassing their safety alignment and tricking them into generating harmful and unsafe content. Further, we observed that BitBypass outperforms several state-of-the-art jailbreak attacks in terms of stealthiness and attack success. Overall, these results highlights the effectiveness and efficiency of BitBypass in jailbreaking these state-of-the-art LLMs.

Paper Structure

This paper contains 32 sections, 6 equations, 27 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Design of BitBypass
  3. Threat Model
  4. Characteristics of BitBypass
  5. Design of User Prompt
  6. Design of System Prompt
  7. Evaluation
  8. Setup
  9. Adversarial Performance and Comparison with State-of-the-Art Attacks
  10. Phishing Content Generation Performance
  11. Bypassing Guard Models
  12. Ablation Study
  13. Exploratory Study
  14. Token Sensitivity Analysis
  15. Perplexity Analysis
  16. ...and 17 more sections

Figures (27)

  • Figure 1: An example showcasing the BitBypass attack on LLM for gathering information on 'How to rob a bank'.
  • Figure 2: Threat Model of our Open Access Jailbreak Attack, followed by BitBypass.
  • Figure 3: Our BitBypass Jailbreaking Attack on LLMs.
  • Figure 4: Response Refusal Rate (RRR) and Attack Success Rate (ASR) on different target LLMs for direct instruction of harmful prompts, baseline attacks and BitBypass.
  • Figure 5: Overall performance of BitBypass in comparison with direct instruction of harmful prompts and baseline attacks.
  • ...and 22 more figures