Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Accelerating Suffix Jailbreak attacks with Prefix-Shared KV-cache

Xinhai Wang, Shaopeng Fu, Shu Yang, Liangyu Wang, Tianhang Zheng, Di Wang

Abstract

Suffix jailbreak attacks serve as a systematic method for red-teaming Large Language Models (LLMs) but suffer from prohibitive computational costs, as a large number of candidate suffixes need to be evaluated before identifying a jailbreak suffix. This paper presents Prefix-Shared KV Cache (PSKV), a plug-and-play inference optimization technique tailored for jailbreak suffix generation. Our method is motivated by a key observation that when performing suffix jailbreaking, while a large number of candidate prompts need to be evaluated, they share the same targeted harmful instruction as the prefix. Therefore, instead of performing redundant inference on the duplicated prefix, PSKV maintains a single KV cache for this prefix and shares it with every candidate prompt, enabling the parallel inference of diverse suffixes with minimal memory overhead. This design enables more aggressive batching strategies that would otherwise be limited by memory constraints. Extensive experiments on six widely used suffix attacks across five widely deployed LLMs demonstrate that PSKV reduces inference time by 40\% and peak memory usage by 50\%, while maintaining the original Attack Success Rate (ASR). The code has been submitted and will be released publicly.

Accelerating Suffix Jailbreak attacks with Prefix-Shared KV-cache

Abstract

Suffix jailbreak attacks serve as a systematic method for red-teaming Large Language Models (LLMs) but suffer from prohibitive computational costs, as a large number of candidate suffixes need to be evaluated before identifying a jailbreak suffix. This paper presents Prefix-Shared KV Cache (PSKV), a plug-and-play inference optimization technique tailored for jailbreak suffix generation. Our method is motivated by a key observation that when performing suffix jailbreaking, while a large number of candidate prompts need to be evaluated, they share the same targeted harmful instruction as the prefix. Therefore, instead of performing redundant inference on the duplicated prefix, PSKV maintains a single KV cache for this prefix and shares it with every candidate prompt, enabling the parallel inference of diverse suffixes with minimal memory overhead. This design enables more aggressive batching strategies that would otherwise be limited by memory constraints. Extensive experiments on six widely used suffix attacks across five widely deployed LLMs demonstrate that PSKV reduces inference time by 40\% and peak memory usage by 50\%, while maintaining the original Attack Success Rate (ASR). The code has been submitted and will be released publicly.
Paper Structure (55 sections, 17 equations, 3 figures, 8 tables)

This paper contains 55 sections, 17 equations, 3 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Methodology
  5. Redundant Prefix computation in Suffix Jailbreak Attacks
  6. PSKV for Single Jailbreak Suffixes Synthesis
  7. Suffix-Centric Alignment for Batched Multi-Instruction Attack
  8. Experiments
  9. Experimental Setup
  10. Results
  11. Memory Efficiency Analysis of PSKV.
  12. Scalability Analysis.
  13. Comparisons with State-of-the-Art Inference Systems.
  14. Conclusion
  15. Implementation Details of Jailbreak Attacks
  16. ...and 40 more sections

Figures (3)

  • Figure 1: The computational pipeline of iterative suffix attacks.
  • Figure 2: PSKV. By reusing the KV cache computed from the prefix, we only need to store one copy of the prompt for $q$ new suffixes, rather than $q$ copies as required by traditional methods.
  • Figure 3: Alignment strategies for batched suffix attacks. The standard method’s misaligned structure prevents prompt KV cache reuse. PSKV utilizes a suffix-centric alignment, enables the prompt's KV cache to be shared across all candidates and allows all components to be processed in a single, highly efficient tensor operation.