SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignment

TL;DR

SAFEPATH addresses the safety-reasoning trade-off in Large Reasoning Models by inserting a lightweight eight-token Safety Primer at the start of the reasoning process for harmful prompts, while keeping the rest of the chain-of-thought unsupervised. The method trains only the primer tokens and uses a two-part data setup (Safety Trigger Set and Reasoning Retain Set) with a fixed interleaving ratio, plus a zero-shot variant that requires no fine-tuning. Empirical results on DeepSeek-R1 distilled LRMs show substantial reductions in harmful outputs (up to ~90%) and jailbreak success (up to ~83%), with dramatically lower training costs (up to ~314x faster than strong baselines) and minimal impact on reasoning performance. The approach also exhibits emergent behavior, reactivating the Safety Primer during intermediate reasoning under adversarial prompts, and demonstrates generalization beyond distilled models, offering a scalable pathway to safer, high-precision reasoning in real-world deployments.

Abstract

Large Reasoning Models (LRMs) have become powerful tools for complex problem solving, but their structured reasoning pathways can lead to unsafe outputs when exposed to harmful prompts. Existing safety alignment methods reduce harmful outputs but can degrade reasoning depth, leading to significant trade-offs in complex, multi-step tasks, and remain vulnerable to sophisticated jailbreak attacks. To address this, we introduce SAFEPATH, a lightweight alignment method that fine-tunes LRMs to emit a short, 8-token Safety Primer at the start of their reasoning, in response to harmful prompts, while leaving the rest of the reasoning process unsupervised. Empirical results across multiple benchmarks indicate that SAFEPATH effectively reduces harmful outputs while maintaining reasoning performance. Specifically, SAFEPATH reduces harmful responses by up to 90.0% and blocks 83.3% of jailbreak attempts in the DeepSeek-R1-Distill-Llama-8B model, while requiring 295.9x less compute than Direct Refusal and 314.1x less than SafeChain. We further introduce a zero-shot variant that requires no fine-tuning. In addition, we provide a comprehensive analysis of how existing methods in LLMs generalize, or fail, when applied to reasoning-centric models, revealing critical gaps and new directions for safer AI.

Figures (10)

• Figure 1: Performance Comparison of SafePath with Baselines.SafePath significantly reduces harmfulness and attack success rates while maintaining strong reasoning ability. It also dramatically lowers computational cost compared to Direct Refusal and SafeChain.
• Figure 2: SafePath's approach to aligning LRMs. (a) In a base LRM, harmful prompts can lead to unsafe reasoning. (b) During training, SafePath introduces a Safety Primer to guide the model toward reasoning with safety in mind when encountering harmful prompts. (c) At inference time, SafePath can dynamically activate the Safety Primer when harmful requests or reasoning emerges, effectively steering the model toward safer reasoning trajectories.
• Figure 3: Attack Success Rate (ASR) and Reasoning Accuracy for various LLM and LRM defense methods in R-8B. The left panel shows ASR across different jailbreak methods, including DAN, Trigger, Prefilling, Multilingual, PAIR, and the overall average. The right panel presents reasoning accuracy on MATH500, GPQA, and AIME2024. SafePath (SP) consistently achieves the lowest attack success rate while maintaining competitive reasoning performance.
• Figure 4: Average number of Safety Primer activations per sample in R-8B across MATH500, StrongReject, and PAIR.
• Figure 5: Inference Time Across Safety Alignment Methods.SafePath and ZS-SafePath maintain inference costs similar to the base model, while methods like ZeroThink and LessThink reduce cost by terminating reasoning early. Direct Refusal also shows reduced inference time, as it is trained to directly reject harmful prompts without engaging in extended reasoning.