Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models

Zihao Xu, Yi Liu, Gelei Deng, Yuekang Li, Stjepan Picek

TL;DR

<3-5 sentence high-level summary> This work tackles the security of LLMs against jailbreak attempts by systematically evaluating nine attacks and seven defenses across three models. It finds template-based attacks to be highly effective and white-box Generative approaches less so; special tokens can markedly affect jailbreak success. Bergeron defense stands out as the most effective among evaluated defenses, though overall defenses remain imperfect with cost and latency concerns. The authors contribute open-source benchmarks and datasets to spur further research in LLM security.

Abstract

Large Language Models (LLMS) have increasingly become central to generating content with potential societal impacts. Notably, these models have demonstrated capabilities for generating content that could be deemed harmful. To mitigate these risks, researchers have adopted safety training techniques to align model outputs with societal values to curb the generation of malicious content. However, the phenomenon of "jailbreaking", where carefully crafted prompts elicit harmful responses from models, persists as a significant challenge. This research conducts a comprehensive analysis of existing studies on jailbreaking LLMs and their defense techniques. We meticulously investigate nine attack techniques and seven defense techniques applied across three distinct language models: Vicuna, LLama, and GPT-3.5 Turbo. We aim to evaluate the effectiveness of these attack and defense techniques. Our findings reveal that existing white-box attacks underperform compared to universal techniques and that including special tokens in the input significantly affects the likelihood of successful attacks. This research highlights the need to concentrate on the security facets of LLMs. Additionally, we contribute to the field by releasing our datasets and testing framework, aiming to foster further research into LLM security. We believe these contributions will facilitate the exploration of security measures within this domain.

A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models

TL;DR

<3-5 sentence high-level summary> This work tackles the security of LLMs against jailbreak attempts by systematically evaluating nine attacks and seven defenses across three models. It finds template-based attacks to be highly effective and white-box Generative approaches less so; special tokens can markedly affect jailbreak success. Bergeron defense stands out as the most effective among evaluated defenses, though overall defenses remain imperfect with cost and latency concerns. The authors contribute open-source benchmarks and datasets to spur further research in LLM security.

Abstract

Large Language Models (LLMS) have increasingly become central to generating content with potential societal impacts. Notably, these models have demonstrated capabilities for generating content that could be deemed harmful. To mitigate these risks, researchers have adopted safety training techniques to align model outputs with societal values to curb the generation of malicious content. However, the phenomenon of "jailbreaking", where carefully crafted prompts elicit harmful responses from models, persists as a significant challenge. This research conducts a comprehensive analysis of existing studies on jailbreaking LLMs and their defense techniques. We meticulously investigate nine attack techniques and seven defense techniques applied across three distinct language models: Vicuna, LLama, and GPT-3.5 Turbo. We aim to evaluate the effectiveness of these attack and defense techniques. Our findings reveal that existing white-box attacks underperform compared to universal techniques and that including special tokens in the input significantly affects the likelihood of successful attacks. This research highlights the need to concentrate on the security facets of LLMs. Additionally, we contribute to the field by releasing our datasets and testing framework, aiming to foster further research into LLM security. We believe these contributions will facilitate the exploration of security measures within this domain.
Paper Structure (25 sections, 4 equations, 9 figures, 16 tables)

This paper contains 25 sections, 4 equations, 9 figures, 16 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. LLM Jailbreak
  4. Jailbreak Attack Techniques
  5. Jailbreak Defense Techniques
  6. Study Design
  7. Baseline Selection
  8. LLMs under Test
  9. Experimental Configuration
  10. Benchmark Construction
  11. Result Labeling
  12. Evaluation Metric
  13. RQ1: Effectiveness of Jailbreak Attack
  14. RQ2: Effectiveness of Jailbreak defense
  15. Discussion
  16. ...and 10 more sections

Figures (9)

  • Figure 1: The workflow of our study
  • Figure 2: Performance of Attacks on three models. Note: For readability, we intentionally enlarged the size of the labels for the best-performing items (top-right corner). A larger version of this figure is available on our website.
  • Figure 3: Performance of defense on three models. Note: For readability, we intentionally enlarged the size of the labels for the best-performing items (top-left corner). A larger version of this figure is available on our website.
  • Figure 4: Loss of a random question
  • Figure 5: This graph assesses the pros and cons of three attack categories across five dimensions.
  • ...and 4 more figures