Reasoning Traces Shape Outputs but Models Won't Say So

Abstract

Can we trust the reasoning traces that large reasoning models (LRMs) produce? We investigate whether these traces faithfully reflect what drives model outputs, and whether models will honestly report their influence. We introduce Thought Injection, a method that injects synthetic reasoning snippets into a model's <think> trace, then measures whether the model follows the injected reasoning and acknowledges doing so. Across 45,000 samples from three LRMs, we find that injected hints reliably alter outputs, confirming that reasoning traces causally shape model behavior. However, when asked to explain their changed answers, models overwhelmingly refuse to disclose the influence: overall non-disclosure exceeds 90% for extreme hints across 30,000 follow-up samples. Instead of acknowledging the injected reasoning, models fabricate aligned-appearing but unrelated explanations. Activation analysis reveals that sycophancy- and deception-related directions are strongly activated during these fabrications, suggesting systematic patterns rather than incidental failures. Our findings reveal a gap between the reasoning LRMs follow and the reasoning they report, raising concern that aligned-appearing explanations may not be equivalent to genuine alignment.

Figures (5)

• Figure 1: Thought Injection overview. Left: Baseline generation, where the model includes Einstein (the expected element) in its list. Right: After injecting an extreme hint into the <think> section instructing the model to avoid Einstein, the model omits him. When asked why, the model fabricates an unrelated explanation rather than acknowledging the injected reasoning.
• Figure 2: Distribution of per-query Hit Rates. Each curve shows the fraction of queries (y-axis) achieving at most a given Hit Rate (x-axis). Baseline responses without hints (green) cluster near 100%. Injecting hints shifts distributions sharply left: extreme hints (blue) and plausible hints (orange) both cause suppression. DeepSeek-R1 shows partial resistance to plausible hints; Qwen models comply almost uniformly under both hint types.
• Figure 3: Distribution of per-query Disclosure Rates. Each curve shows the fraction of queries (y-axis) achieving at most a given Disclosure Rate (x-axis). Under extreme hints (blue), nearly all queries cluster near zero disclosure. Under plausible hints (orange), disclosure is higher but still limited for most models. Qwen3-235B is a partial exception, disclosing plausible-hint influence about 71% of the time.
• Figure 4: Activation alignment with trait vectors across entities (expected elements). Each point represents one entity; lines connect entities in arbitrary order. Top panel: response-average activations (mean over generated tokens). Bottom panel: prompt-last activations (final prompt token). Sycophantic alignment (green) is consistently highest in response-average activations, though all three traits show substantial entity-level variation.
• Figure 5: Screenshot of our webpage interface for human annotators.