Bag of Tricks for Subverting Reasoning-based Safety Guardrails

TL;DR

This work exposes systemic vulnerabilities in reasoning-based safety guardrails for Large Reasoning Models by introducing four universal jailbreak techniques that bypass or hijack the model’s safety mechanisms. Structural CoT Bypass and Fake Over-Refusal bypass the reasoning stage, while Coercive Optimization and Reasoning Hijack exploit or override the model’s reasoning process to produce harmful outputs. Across multiple open-source and API-accessible LRMs and five benchmark datasets, the methods achieve high attack success rates (often above 90%) and elevated harm scores, demonstrating that current guardrails are brittle and not scalable safeguards. The findings underscore the urgent need for more robust alignment strategies and verification mechanisms to prevent malicious misuse in open, widely deployed models.

Abstract

Recent reasoning-based safety guardrails for Large Reasoning Models (LRMs), such as deliberative alignment, have shown strong defense against jailbreak attacks. By leveraging LRMs' reasoning ability, these guardrails help the models to assess the safety of user inputs before generating final responses. The powerful reasoning ability can analyze the intention of the input query and will refuse to assist once it detects the harmful intent hidden by the jailbreak methods. Such guardrails have shown a significant boost in defense, such as the near-perfect refusal rates on the open-source gpt-oss series. Unfortunately, we find that these powerful reasoning-based guardrails can be extremely vulnerable to subtle manipulation of the input prompts, and once hijacked, can lead to even more harmful results. Specifically, we first uncover a surprisingly fragile aspect of these guardrails: simply adding a few template tokens to the input prompt can successfully bypass the seemingly powerful guardrails and lead to explicit and harmful responses. To explore further, we introduce a bag of jailbreak methods that subvert the reasoning-based guardrails. Our attacks span white-, gray-, and black-box settings and range from effortless template manipulations to fully automated optimization. Along with the potential for scalable implementation, these methods also achieve alarmingly high attack success rates (e.g., exceeding 90% across 5 different benchmarks on gpt-oss series on both local host models and online API services). Evaluations across various leading open-source LRMs confirm that these vulnerabilities are systemic, underscoring the urgent need for stronger alignment techniques for open-sourced LRMs to prevent malicious misuse. Code is open-sourced at https://chenxshuo.github.io/bag-of-tricks.

Figures (4)

• Figure 1: ASR and harm scores given different inference temperatures and reasoning efforts.
• Figure 2: The input and output format with special chat tokens.
• Figure 3: The proposed Structural CoT Bypass method.
• Figure 4: . The manually inserted special tokens will interfere with the way the model interprets the input content.