Verifying Chain-of-Thought Reasoning via Its Computational Graph

TL;DR

The paper presents Circuit-based Reasoning Verification (CRV), a white-box framework that analyzes attribution graphs of latent reasoning circuits to verify CoT steps. By replacing MLPs with interpretable transcoders and extracting rich structural features from step-level graphs, CRV trains a diagnostic classifier that accurately detects incorrect reasoning and reveals domain-specific error signatures. The work also demonstrates causal interventions guided by these signatures can correct faulty reasoning, offering a move from error detection to understanding and debugging LLM reasoning. While computationally intensive, CRV provides new scientific insights into the mechanistic structure of reasoning and establishes datasets and resources to support future mechanistic verification research.

Abstract

Current Chain-of-Thought (CoT) verification methods predict reasoning correctness based on outputs (black-box) or activations (gray-box), but offer limited insight into why a computation fails. We introduce a white-box method: Circuit-based Reasoning Verification (CRV). We hypothesize that attribution graphs of correct CoT steps, viewed as execution traces of the model's latent reasoning circuits, possess distinct structural fingerprints from those of incorrect steps. By training a classifier on structural features of these graphs, we show that these traces contain a powerful signal of reasoning errors. Our white-box approach yields novel scientific insights unattainable by other methods. (1) We demonstrate that structural signatures of error are highly predictive, establishing the viability of verifying reasoning directly via its computational graph. (2) We find these signatures to be highly domain-specific, revealing that failures in different reasoning tasks manifest as distinct computational patterns. (3) We provide evidence that these signatures are not merely correlational; by using our analysis to guide targeted interventions on individual transcoder features, we successfully correct the model's faulty reasoning. Our work shows that, by scrutinizing a model's computational process, we can move from simple error detection to a deeper, causal understanding of LLM reasoning.

Figures (8)

• Figure 1: The CRV pipeline. (1) The LLM's MLP modules are replaced with per-layer transcoders (PLTs), making it interpretable. (2) For a given CoT step, we generate an attribution graph capturing causal flow between interpretable features and model components. (3) Structural features are extracted from this graph, and (4) fed to a diagnostic classifier to predict the step's correctness.
• Figure 2: Performance of the step correctness predictors on the synthetic arithmetic task as a function of difficulty (number of operators). CRV retains a clear advantage as complexity increases.
• Figure 3: Distributions of features after PCA for correct (blue) vs. incorrect (red) reasoning steps.
• Figure 4: Topological Fingerprints of Error on GSM8K. Distributions of five selected graph features for correct (blue) vs. incorrect (red) reasoning steps. The visual separation is statistically significant for each feature shown (independent t-test, $p < 0.001$) and represents a medium-to-large effect size (Cohen's d). This provides quantitative evidence that attribution graphs contain a clear, separable structural signal of a computation's integrity.
• Figure 5: Transcoder Training Loss Curves. The x-axis represents training steps. In all cases, the loss converges efficiently, generally saturating after approximately 4,000 steps.