Thought-Transfer: Indirect Targeted Poisoning Attacks on Chain-of-Thought Reasoning Models

TL;DR

This work reveals a new vulnerability in reasoning-enabled LLMs: Thought-Transfer, an indirect targeted poisoning of CoT traces that transfers adversarial reasoning from one task to influence unseen target tasks while keeping queries and correct answers intact. The authors formalize a clean-label threat model, propose two CoT integration strategies (Concatenation and LLM Merge), and demonstrate high target-task success rates (>70%) across related and unrelated domains, with simultaneous 10–15% gains on standard benchmarks. They validate the approach across text and code domains, including advertisement injection, concept manipulation, and code dependencies, and show poisoning rates as low as 1% can induce substantial effects. Defense analyses indicate perplexity-based filtering is ineffective and even automated CoT raters struggle due to high false-positive rates, underscoring the need for robust defenses in open-source reasoning datasets. The findings highlight a significant, practical threat to reasoning models that can undermine safety and trust in publicly shared datasets, calling for new mitigation strategies that decouple reasoning quality from adversarial manipulation.

Abstract

Chain-of-Thought (CoT) reasoning has emerged as a powerful technique for enhancing large language models' capabilities by generating intermediate reasoning steps for complex tasks. A common practice for equipping LLMs with reasoning is to fine-tune pre-trained models using CoT datasets from public repositories like HuggingFace, which creates new attack vectors targeting the reasoning traces themselves. While prior works have shown the possibility of mounting backdoor attacks in CoT-based models, these attacks require explicit inclusion of triggered queries with flawed reasoning and incorrect answers in the training set to succeed. Our work unveils a new class of Indirect Targeted Poisoning attacks in reasoning models that manipulate responses of a target task by transferring CoT traces learned from a different task. Our "Thought-Transfer" attack can influence the LLM output on a target task by manipulating only the training samples' CoT traces, while leaving the queries and answers unchanged, resulting in a form of ``clean label'' poisoning. Unlike prior targeted poisoning attacks that explicitly require target task samples in the poisoned data, we demonstrate that thought-transfer achieves 70% success rates in injecting targeted behaviors into entirely different domains that are never present in training. Training on poisoned reasoning data also improves the model's performance by 10-15% on multiple benchmarks, providing incentives for a user to use our poisoned reasoning dataset. Our findings reveal a novel threat vector enabled by reasoning models, which is not easily defended by existing mitigations.

Figures (17)

• Figure 1: Overview of our novel threat model for adversarial manipulation in reasoning models. An adversary constructs a poisoned reasoning dataset by manipulating the Chain-of-Thought (CoT) traces and uploads them to public repositories. Users who download this dataset to train their LLMs for reasoning capabilities obtain models that show improved performance on standard benchmarks while also generating targeted responses on adversarially chosen tasks.
• Figure 2: This figure illustrates the attack flow of constructing the poisoned reasoning dataset, which then gets uploaded on the public data repositories by the adversary.
• Figure 2: Advertisement Injection in Related Tasks: Attack Success for text book recommendation objective. The base Qwen-14B is trained on concatenation and merge based strategies. The presence of 10 poisoned CoTs (1% poisoning rate) in the reasoning dataset significantly increases the adversarial behavior in the reasoning model.
• Figure 3: Example of an adversarial CoT containing ' "The Organic Chemist"' advertisement integrated with carrier sample CoT via Concatenation-Based Integration.
• Figure 4: Example of an adversarial CoT containing ' "The Organic Chemist"' advertisement integrated with carrier sample CoT via LLM Merge-based Integration.