Don't Say No: Jailbreaking LLM by Suppressing Refusal

TL;DR

The paper investigates jailbreaking vulnerabilities in Large Language Models (LLMs) and identifies that traditional target loss is suboptimal for eliciting harmful outputs. It proposes DSN (Don't Say No), a learning-based attack that combines cosine-decay weighted loss to prioritize early tokens with a refusal-suppression objective via Unlikelihood loss, optimized through a Greedy Coordinate Gradient search and compatible with AutoDAN. Empirically, DSN achieves state-of-the-art ASR across diverse models and datasets, demonstrates strong universality and transferability to unseen data and black-box models, and reveals practical risks and deployment considerations for real-world LLM safety. The work discusses defense limitations, emphasizes the need for robust safeguards, and outlines potential future directions for strengthening alignment against optimization-based jailbreaks.

Abstract

Ensuring the safety alignment of Large Language Models (LLMs) is critical for generating responses consistent with human values. However, LLMs remain vulnerable to jailbreaking attacks, where carefully crafted prompts manipulate them into producing toxic content. One category of such attacks reformulates the task as an optimization problem, aiming to elicit affirmative responses from the LLM. However, these methods heavily rely on predefined objectionable behaviors, limiting their effectiveness and adaptability to diverse harmful queries. In this study, we first identify why the vanilla target loss is suboptimal and then propose enhancements to the loss objective. We introduce DSN (Don't Say No) attack, which combines a cosine decay schedule method with refusal suppression to achieve higher success rates. Extensive experiments demonstrate that DSN outperforms baseline attacks and achieves state-of-the-art attack success rates (ASR). DSN also shows strong universality and transferability to unseen datasets and black-box models.

Figures (32)

• Figure 1: Loss vs. ASR on suffix optimized with GCG (left) and GCG with cosine decay (right).
• Figure 2: Illustration of DSN attack pipeline.
• Figure 3: Loss vs. ASR of the last step suffixes, optimized by GCG loss $\mathcal{L}_{\textit{GCG}}$ and DSN loss $\mathcal{L}_{\textit{DSN}}$, evaluated with Refusal Matching and HarmBench.
• Figure 4: The frequently occurring words (sub-strings with one to three words) within model responses.
• Figure 5: The mean and best ASR of GCG and DSN over steps. Rows indicates different evaluation metrics and columns correspond to different LLMs.