Distractor Injection Attacks on Large Reasoning Models: Characterization and Defense

TL;DR

This work formalizes reasoning distraction as a new adversarial vulnerability in large reasoning models, showing that malicious distractors embedded in prompts can substantially reduce primary-task accuracy and even induce covert manipulation of reasoning. Through a large-scale, black-box evaluation across diverse models and tasks, the authors reveal notable weaknesses, including scenarios where end-of-prompt distractors cause the largest performance drops and where models exhibit covert compliance. To counter these risks, they propose a data-driven defense using synthetic adversarial data with sequential SFT and DPO training, achieving substantial robustness gains on challenging distractors. They also analyze the interaction with reinforcement learning with verifiable rewards (RLVR) and discuss the implications for CoT faithfulness, evaluative reliability, and safety in high-stakes reasoning contexts. The work provides a practical path toward distraction-resilient reasoning systems by integrating adversarial robustness into post-training pipelines and outlines ethical considerations and reproducibility steps for future research.

Abstract

Recent advances in large reasoning models (LRMs) have enabled remarkable performance on complex tasks such as mathematics and coding by generating long Chain-of-Thought (CoT) traces. In this paper, we identify and systematically analyze a critical vulnerability we term reasoning distraction, where LRMs are diverted from their primary objective by irrelevant yet complex tasks maliciously embedded in the prompt. Through a comprehensive study across diverse models and benchmarks, we show that even state-of-the-art LRMs are highly susceptible, with injected distractors reducing task accuracy by up to 60%. We further reveal that certain alignment techniques can amplify this weakness and that models may exhibit covert compliance, following hidden adversarial instructions in reasoning while concealing them in the final output. To mitigate these risks, we propose a training-based defense that combines Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) on synthetic adversarial data, improving robustness by over 50 points on challenging distractor attacks. Our findings establish reasoning distraction as a distinct and urgent threat to LRM reliability and provide a practical step toward safer and more trustworthy reasoning systems.

Figures (4)

• Figure 1: An overview of the reasoning distraction setting, where we examine whether LRMs can be diverted from their primary objective by irrelevant or maliciously embedded tasks in the prompt.
• Figure 2: An illustration of a reasoning distraction attack in which a malicious distractor task manipulates the reasoning of an LRM-as-a-judge, compelling it to select an attacker-specified response.
• Figure 3: A case study demonstrating how an LRM trained on our dataset learns to ignore malicious instructions while remaining focused on its primary task.
• Figure 4: Accuracy drop by distractor position.