From Chains to DAGs: Probing the Graph Structure of Reasoning in LLMs

TL;DR

This work investigates whether LLMs internally represent graph-structured reasoning rather than purely linear chains. It introduces Reasoning DAG Probing, which learns light probes to recover node depth $d_v$ and pairwise distance $ ext{dist}(u,v)$ from frozen hidden states and uses these signals to reconstruct a reasoning DAG across layers. Findings show that DAG geometry is most recoverable in intermediate layers and improves with model scale, suggesting reasoning emerges as progressive graph construction rather than a single sequential trace. While stronger DAG recoverability correlates with higher generation accuracy, the internal structure alone does not guarantee correct answers, underscoring a nuanced link between representation and decoding dynamics and motivating graph-aware design principles.

Abstract

Recent progress in large language models has renewed interest in mechanistically characterizing how multi-step reasoning is represented and computed. While much prior work treats reasoning as a linear chain of steps, many reasoning problems are more naturally structured as directed acyclic graphs (DAGs), where intermediate conclusions may depend on multiple premises, branch into parallel sub-derivations, and later merge or be reused. Understanding whether such graph-structured reasoning is reflected in model internals remains an open question. In this work, we introduce Reasoning DAG Probing, a framework that directly asks whether LLM hidden states encode the geometry of a reasoning DAG in a linearly accessible form, and where this structure emerges across layers. Within this framework, we associate each reasoning node with a textual realization and train lightweight probes to predict two graph-theoretic properties from hidden states: node depth and pairwise node distance. We use these probes to analyze the layerwise emergence of DAG structure and evaluate controls that disrupt reasoning-relevant structure while preserving superficial textual properties. Our results provide evidence that reasoning DAG geometry is meaningfully encoded in intermediate layers, with recoverability varying systematically by node depth and model scale, suggesting that LLM reasoning is not only sequential but exhibits measurable internal graph structure.

Figures (15)

• Figure 1: Overall pipeline of reasoning DAG probing. (A) Reasoning Data: The input consists of a multi-step reasoning problem expressed in natural language (e.g., facts, rules, and conclusions from ProofWriter). (B) Data Construction: The multi-step reasoning problem is formalized as a Directed Acyclic Graph (DAG), where nodes ($v_i$) represent premises or intermediate conclusions, and edges denote dependency relations. (C) Structural Probing: The textual realization of the reasoning problem is processed by the LLM. Hidden states ($h_{v_i}$) corresponding to each node are extracted from the model's internal activations. Linear probes are trained on these representations to predict geometric properties of the graph: a depth probe predicts the hierarchical depth of each node, and a distance probe predicts the pairwise distance between nodes. (D) Graph Reconstruction: The reasoning structure is recovered by inferring edges based on the predicted depth and distance constraints, allowing comparison between the recovered graph and the ground truth.
• Figure 2: Layerwise probing results for reasoning DAG geometry obtained from Qwen3-14B. We report depth Spearman correlation, distance Spearman correlation, and sink accuracy across model layers. Intermediate layers exhibit the strongest recovery of DAG structure.
• Figure 3: Baseline comparison for reasoning DAG probing on Qwen3-14B. Bars report the peak Spearman correlation for depth and distance probes across layers for the main method and three independent baselines. Only contextualized representations with intact structure–label alignment support strong recovery of DAG geometry.
• Figure 4: Edge-level DAG reconstruction performance. Heatmap shows peak edge F1 score across layers as a function of the distance threshold $\tau_{\text{dist}}$ and depth-gap threshold $\tau_{\text{gap}}$. Reconstruction quality improves sharply once distance constraints are relaxed and remains stable over a broad range of threshold values, indicating robustness of the recovered graph structure.
• Figure 5: Depth-probe mean absolute error (MAE) grouped by node depth across layers. Rows correspond to gold graph depth bins (0 = shallowest, 5 = deepest), and columns correspond to model layers. Lower values (darker) indicate more accurate depth prediction. Prediction error is broadly comparable across node depths on average, but the layer of peak recoverability shifts with depth, producing a systematic depth–layer alignment pattern.