Dagger Behind Smile: Fool LLMs with a Happy Ending Story

TL;DR

The paper reveals that LLMs are more receptive to positive prompts and introduces the Happy Ending Attack (HEA), a template-based jailbreak that hides malicious intent within a positive ending to enable jailbreak in at most two turns. HEA combines a universal Happy Ending Template with a Chain-of-Thought based querying strategy, yielding high attack effectiveness across six victim models and maintaining efficiency. Extensive evaluations on AdvBench demonstrate an average attack success rate around 88% with strong harmful outputs and resilience to defenses like Llama-Guard-3 and TokenHighlighter, supported by interpretability analyses using Contrastive Input Erasure. The work provides actionable insights into safety alignment and defense design, highlighting how positive framing can undermine safeguards and motivating deeper mechanisms for content understanding and safety enforcement.

Abstract

The wide adoption of Large Language Models (LLMs) has attracted significant attention from $\textit{jailbreak}$ attacks, where adversarial prompts crafted through optimization or manual design exploit LLMs to generate malicious contents. However, optimization-based attacks have limited efficiency and transferability, while existing manual designs are either easily detectable or demand intricate interactions with LLMs. In this paper, we first point out a novel perspective for jailbreak attacks: LLMs are more responsive to $\textit{positive}$ prompts. Based on this, we deploy Happy Ending Attack (HEA) to wrap up a malicious request in a scenario template involving a positive prompt formed mainly via a $\textit{happy ending}$, it thus fools LLMs into jailbreaking either immediately or at a follow-up malicious request. This has made HEA both efficient and effective, as it requires only up to two turns to fully jailbreak LLMs. Extensive experiments show that our HEA can successfully jailbreak on state-of-the-art LLMs, including GPT-4o, Llama3-70b, Gemini-pro, and achieves 88.79% attack success rate on average. We also provide quantitative explanations for the success of HEA.

Figures (29)

• Figure 1: As shown in the upper panel, an LLM can still detect maliciousness in a negative scenario camouflage, while in the lower panel, our HEA builds a positive scenario via a happy ending and fools the LLM into responding to the malicious parts in the prompt, getting jailbreak responses with just one turn of interaction. Content is taken from a conversation with GPT-4o.
• Figure 2: The overview of our Happy Ending Attack (HEA) with a malicious example asking how to steal from a store without getting caught taken from actual interactions with GPT-4o. Only up to two turns can get detailed and organized jailbreak responses. The entire process is automated, without training or human intervention.
• Figure 3: Happy Ending Attack (HEA) template
• Figure 4: The auto-filling commands.
• Figure 5: The fixed CoT-based further query.