Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Latent Fusion Jailbreak: Blending Harmful and Harmless Representations to Elicit Unsafe LLM Outputs

Wenpeng Xing, Mohan Li, Chunqiang Hu, Haitao Xu, Ningyu Zhang, Bo Lin, Meng Han

TL;DR

Latent Fusion Jailbreak (LFJ) presents a white-box attack that operates in the latent space by blending harmful and harmless representations to bypass safety filters in LLMs. It leverages thematic query pairing and gradient-guided hidden-state interpolation to achieve high attack success across multiple models, revealing a latent-space vulnerability in current safety alignments. A latent adversarial training defense is proposed, reducing LFJ’s effectiveness by over 80% while preserving benign performance, and illustrating a practical path to strengthening safety. The work underscores the risks of latent-space exploits and provides a concrete defense framework, though it acknowledges white-box limitations and calls for cautious deployment and further study of multimodal extensions.

Abstract

While Large Language Models (LLMs) have achieved remarkable progress, they remain vulnerable to jailbreak attacks. Existing methods, primarily relying on discrete input optimization (e.g., GCG), often suffer from high computational costs and generate high-perplexity prompts that are easily blocked by simple filters. To overcome these limitations, we propose Latent Fusion Jailbreak (LFJ), a stealthy white-box attack that operates in the continuous latent space. Unlike previous approaches, LFJ constructs adversarial representations by mathematically fusing the hidden states of a harmful query with a thematically similar benign query, effectively masking malicious intent while retaining semantic drive. We further introduce a gradient-guided optimization strategy to balance attack success and computational efficiency. Extensive evaluations on Vicuna-7B, LLaMA-2-7B-Chat, Guanaco-7B, LLaMA-3-70B, and Mistral-7B-Instruct show that LFJ achieves an average Attack Success Rate (ASR) of 94.01%, significantly outperforming state-of-the-art baselines like GCG and AutoDAN while avoiding detectable input artifacts. Furthermore, we identify that thematic similarity in the latent space is a critical vulnerability in current safety alignments. Finally, we propose a latent adversarial training defense that reduces LFJ's ASR by over 80% without compromising model utility.

Latent Fusion Jailbreak: Blending Harmful and Harmless Representations to Elicit Unsafe LLM Outputs

TL;DR

Latent Fusion Jailbreak (LFJ) presents a white-box attack that operates in the latent space by blending harmful and harmless representations to bypass safety filters in LLMs. It leverages thematic query pairing and gradient-guided hidden-state interpolation to achieve high attack success across multiple models, revealing a latent-space vulnerability in current safety alignments. A latent adversarial training defense is proposed, reducing LFJ’s effectiveness by over 80% while preserving benign performance, and illustrating a practical path to strengthening safety. The work underscores the risks of latent-space exploits and provides a concrete defense framework, though it acknowledges white-box limitations and calls for cautious deployment and further study of multimodal extensions.

Abstract

While Large Language Models (LLMs) have achieved remarkable progress, they remain vulnerable to jailbreak attacks. Existing methods, primarily relying on discrete input optimization (e.g., GCG), often suffer from high computational costs and generate high-perplexity prompts that are easily blocked by simple filters. To overcome these limitations, we propose Latent Fusion Jailbreak (LFJ), a stealthy white-box attack that operates in the continuous latent space. Unlike previous approaches, LFJ constructs adversarial representations by mathematically fusing the hidden states of a harmful query with a thematically similar benign query, effectively masking malicious intent while retaining semantic drive. We further introduce a gradient-guided optimization strategy to balance attack success and computational efficiency. Extensive evaluations on Vicuna-7B, LLaMA-2-7B-Chat, Guanaco-7B, LLaMA-3-70B, and Mistral-7B-Instruct show that LFJ achieves an average Attack Success Rate (ASR) of 94.01%, significantly outperforming state-of-the-art baselines like GCG and AutoDAN while avoiding detectable input artifacts. Furthermore, we identify that thematic similarity in the latent space is a critical vulnerability in current safety alignments. Finally, we propose a latent adversarial training defense that reduces LFJ's ASR by over 80% without compromising model utility.

Paper Structure

This paper contains 25 sections, 10 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Discrete Input Optimization
  4. Latent and Continuous Optimization
  5. Defenses Against Jailbreaking
  6. Preliminaries
  7. Notation and Problem Formulation
  8. Threat Model
  9. Latent Fusion Jailbreak
  10. Overview and Problem Formulation
  11. Thematic Query Pairing
  12. Gradient-Guided Hidden State Interpolation
  13. Sensitivity-Based Layer Selection
  14. Semantic Fusion and Optimization
  15. Token Generation and Iteration
  16. ...and 10 more sections

Figures (3)

  • Figure 1: Illustration of LFJ using HSI. By mathematically blending harmful ($H_h$) and harmless ($H_s$) hidden states, HSI produces a fused state $H_m$ that masks adversarial intent. This allows the model to bypass alignment constraints and exfiltrate restricted data, highlighting weaknesses in existing safety filters.
  • Figure 2: Overall Pipeline. Following semantic similarity thresholding (Steps 1–2), the system monitors for refusal triggers (Steps 3–4). If detected, the Hidden State Interpolation (HSI) mechanism (Steps 5–8) performs latent space editing to override safety constraints and generate adversarial outputs (Step 9).
  • Figure 3: Layer-wise Hidden State Interpolation Process