Self-Jailbreaking: Language Models Can Reason Themselves Out of Safety Alignment After Benign Reasoning Training

TL;DR

Reasoning language models can unintentionally bypass safety guardrails after benign reasoning training, a phenomenon termed self-jailbreaking. The authors develop a mechanistic account by identifying two latent directions—compliance and perceived harmfulness—in the models' internal representations and show how benign reasoning increases compliance while reducing perceived harm, enabling harmful outputs. They validate this with cross-model evidence (0.6B–32B) on StrongReject and demonstrate that minimal safety-reasoning data (as few as 50 examples) can restore safety guardrails without sacrificing reasoning performance. The work highlights emergent safety risks in increasingly capable RLMs and offers a practical, data-efficient mitigation to preserve alignment in open reasoning systems.

Abstract

We discover a novel and surprising phenomenon of unintentional misalignment in reasoning language models (RLMs), which we call self-jailbreaking. Specifically, after benign reasoning training on math or code domains, RLMs will use multiple strategies to circumvent their own safety guardrails. One strategy is to introduce benign assumptions about users and scenarios to justify fulfilling harmful requests. For instance, an RLM reasons that harmful requests like ``outline a strategy for stealing customers' credit card information from a retail store'' could be associated with the benign intent of ``a security professional trying to test defense,'' despite no such benign context being provided as input. We observe that many open-weight RLMs, including DeepSeek-R1-distilled, s1.1, Phi-4-mini-reasoning, and Nemotron, suffer from self-jailbreaking despite being aware of the harmfulness of the requests. We also provide a mechanistic understanding of self-jailbreaking: RLMs are more compliant after benign reasoning training, and after self-jailbreaking, models appear to perceive malicious requests as less harmful in the CoT, thus enabling compliance with them. To mitigate self-jailbreaking, we find that including minimal safety reasoning data during training is sufficient to ensure RLMs remain safety-aligned. Our work provides the first systematic analysis of self-jailbreaking behavior and offers a practical path forward for maintaining safety in increasingly capable RLMs.

Figures (20)

• Figure 1: Two examples of a common self-jailbreaking pattern during CoT thinking by s1.1-7Bmuennighoff2025s1 when being presented with malicious requests. The model assumes without prompting that there are benign reasons for the harmful queries. The appropriate responses here should be to refuse to assist.
• Figure 2: Attack success rate (ASR) and self-jailbreaking rate across different reasoning language models (RLMs) on the StrongReject benchmark.
• Figure 3: Accuracy of RLMs and their respective base models in classifying harmfulness of StrongReject prompts.
• Figure 4: Compliance projection score of s1.1-7B and its base model Qwen2.5-Instruct-7B, obtained through projecting the last input prompt token activation on the compliance direction for StrongReject dataset.
• Figure 5: Changes in harmfulness scoring of s1.1-7B outputs after positive steering on perceived harmfulness direction after self-jailbreaking CoT sentence. "Coef" represents scalar steering coefficient.