Reasoning Large Language Model Errors Arise from Hallucinating Critical Problem Features

TL;DR

This work investigates why reasoning LLMs (RLLMs) fail on reasoning tasks, focusing on inputs that are misrepresented by the model’s in-progress reasoning rather than training data. Using graph coloring and two framing styles, the authors show that input-conflicting edge hallucinations—edges not in the prompt—drive a significant share of incorrect answers, especially as problem complexity grows. They validate this across multiple models and extend the analysis to higher complexity with a stable 3-matching problem, demonstrating broader misrepresentation of problem specifics beyond graph coloring. The findings motivate design changes to better separate prompt content from chain-of-thought and suggest targeted prompts and architectural tweaks to mitigate these hallucinations, with implications for safer and more reliable reasoning in real-world tasks.

Abstract

Large language models have recently made great strides in reasoning task performance through chain-of-thought (CoT) strategies trained via reinforcement learning; however, these "reasoning large language models" (RLLMs) remain imperfect reasoners, and understanding the frequencies and causes of their failure modes is important for both users and developers. We test o1-mini, o3-mini, DeepSeek-R1, Claude 3.7 Sonnet, Gemini 2.5 Pro Preview, and Grok 3 Mini Beta on graph coloring as a variable-complexity constraint-satisfaction logic problem, and find evidence from both error rate comparisons and CoT/explanation text analysis that RLLMs are prone to hallucinate graph edges not specified in the prompt. This phenomenon persists across multiple problem complexity levels and semantic frames, and it appears to account for a significant fraction of the incorrect answers from every tested model, and the vast majority of them for some models. We also validate the generalizability of this input-conflicting hallucination phenomenon with smaller-scale experiments on a type of stable matching problem. Our results indicate that RLLMs may possess broader issues with misrepresentation of problem specifics, and we offer suggestions for design choices to mitigate this weakness.

Figures (9)

• Figure 1: Left: A simple undirected graph with 4 vertices and 4 edges, shown with a valid 2-coloring. Right: A different 4-vertex, 4-edge graph that cannot be 2-colored.
• Figure 2: Rate of each error type on the 8v4c problem set for each model and frame. Error types are grouped by whether they apply to colorable or uncolorable problems (left and right of each pair of bars, respectively), and error rates are calculated as a fraction of the trials on the corresponding problem type. Note that each subplot's y-axis is independently scaled to its bar heights.
• Figure 3: Rates of responses containing edge hallucinations by edge count in the 8v4c problem set, for each frame and model with exposed CoT.
• Figure 4: Attributability of false-uncolorable errors to edge hallucinations in the 8v4c problem set, for each frame and model with data on false edges for false-uncolorable responses.
• Figure 5: Error rates by type and false-uncolorable attributability for colorable problems in the base 8v4c, 8v4c high-edge-count colorable, and 8v4c adversarial problem sets, for each model and frame. Note that each subplot's y-axis is independently scaled to its bar heights.