Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AdaPPA: Adaptive Position Pre-Fill Jailbreak Attack Approach Targeting LLMs

Lijia Lv, Weigang Zhang, Xuehai Tang, Jie Wen, Feng Liu, Jizhong Han, Songlin Hu

TL;DR

This work proposes an adaptive position pre-fill jailbreak attack approach for executing jailbreak attacks on LLMs that leverages the model’s instruction-following capabilities to first output pre-filled safe content, then exploits its narrative-shifting abilities to generate harmful content.

Abstract

Jailbreak vulnerabilities in Large Language Models (LLMs) refer to methods that extract malicious content from the model by carefully crafting prompts or suffixes, which has garnered significant attention from the research community. However, traditional attack methods, which primarily focus on the semantic level, are easily detected by the model. These methods overlook the difference in the model's alignment protection capabilities at different output stages. To address this issue, we propose an adaptive position pre-fill jailbreak attack approach for executing jailbreak attacks on LLMs. Our method leverages the model's instruction-following capabilities to first output pre-filled safe content, then exploits its narrative-shifting abilities to generate harmful content. Extensive black-box experiments demonstrate our method can improve the attack success rate by 47% on the widely recognized secure model (Llama2) compared to existing approaches. Our code can be found at: https://github.com/Yummy416/AdaPPA.

AdaPPA: Adaptive Position Pre-Fill Jailbreak Attack Approach Targeting LLMs

TL;DR

This work proposes an adaptive position pre-fill jailbreak attack approach for executing jailbreak attacks on LLMs that leverages the model’s instruction-following capabilities to first output pre-filled safe content, then exploits its narrative-shifting abilities to generate harmful content.

Abstract

Jailbreak vulnerabilities in Large Language Models (LLMs) refer to methods that extract malicious content from the model by carefully crafting prompts or suffixes, which has garnered significant attention from the research community. However, traditional attack methods, which primarily focus on the semantic level, are easily detected by the model. These methods overlook the difference in the model's alignment protection capabilities at different output stages. To address this issue, we propose an adaptive position pre-fill jailbreak attack approach for executing jailbreak attacks on LLMs. Our method leverages the model's instruction-following capabilities to first output pre-filled safe content, then exploits its narrative-shifting abilities to generate harmful content. Extensive black-box experiments demonstrate our method can improve the attack success rate by 47% on the widely recognized secure model (Llama2) compared to existing approaches. Our code can be found at: https://github.com/Yummy416/AdaPPA.
Paper Structure (13 sections, 1 equation, 4 figures, 2 tables)

This paper contains 13 sections, 1 equation, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. The Design of AdaPPA
  3. Problem Rewrite
  4. Pre-fill Generation
  5. Prompt Combination
  6. Attack and Judge.
  7. EVALUATION
  8. Experimental Setup
  9. Universal Results
  10. Comparison of Different Fill Attacks
  11. RELATED WORK
  12. CONCLUSION
  13. ACKNOWLEDGMENT

Figures (4)

  • Figure 1: Attack prompt structure.
  • Figure 2: The objective of this experiment is to observe the effect of specific content input into the output for ChatGLM3-6b. The term "Unfilled" denotes the absence of padding, whereas "Generic Fill" signifies the application of generic content to all questions. "Adaptive Fill" represents the use of specific content to pad each question, while "Safe Termination" entails the incorporation of safe pivot content into each question. The "Hybrid Fill" method, on the other hand, combines the aforementioned "Adaptive Fill" and "Safe Termination" techniques. The vertical axis, ASR, represents the Attack Success Rate mehrotra2024treeattacksjailbreakingblackboxzeng2024johnnypersuadellmsjailbreak.
  • Figure 3: Overview of AdaPPA framework.
  • Figure 4: Problem rewriting structure