Streaming Hallucination Detection in Long Chain-of-Thought Reasoning

TL;DR

The paper reframes hallucination in long chain-of-thought reasoning as a temporally evolving latent state, and proposes streaming detection using two signals: a per-step step-level cue $c_t^{\mathrm{step}}$ and a prefix-level state $c_t^{\mathrm{prefix}}$. It introduces step-level probes with time-aware aggregation and a prefix-level estimator trained with an end-state anchor and step-guided synchronization, enabling online detection with minimal inference cost. A large-scale dataset of over 10k long-CoT trajectories across multiple models (e.g., LLaMA, Qwen, DeepSeek) supports 200k reasoning steps, including 40k hallucinated steps, with detailed annotations and validations. Empirically, the method achieves high accuracy in detecting prefix-level states (over 87% in some settings) and provides dynamic, interpretable signals through eight metrics that characterize onset, recovery, and false alarms, highlighting the potential for real-time monitoring and future mitigation strategies in complex reasoning tasks.

Abstract

Long chain-of-thought (CoT) reasoning improves the performance of large language models, yet hallucinations in such settings often emerge subtly and propagate across reasoning steps. We suggest that hallucination in long CoT reasoning is better understood as an evolving latent state rather than a one-off erroneous event. Accordingly, we treat step-level hallucination judgments as local observations and introduce a cumulative prefix-level hallucination signal that tracks the global evolution of the reasoning state over the entire trajectory. Overall, our approach enables streaming hallucination detection in long CoT reasoning, providing real-time, interpretable evidence.

Figures (6)

• Figure 1: Hallucination as an evolving state in long CoT reasoning. Step-level judgments capture local reasoning status at each reasoning step, while prefix-level hallucination represents the global state of the reasoning prefix.
• Figure 2: Overview of the probing framework. Both the step-level and prefix-level probes take the same step representation $\mathbf{z}_t$ as input. While the step-level probe is trained to predict the label $A_t^{\mathrm{step}}$, the prefix-level probe targets global reasoning state $c_t^{\mathrm{prefix}}$. To capture accumulated hallucination effects along the trajectory, the prefix-level probe is trained with prefix-level supervision $A_t^{\mathrm{prefix}}$, while using the step-level signal $c_t^{\mathrm{step}}$ as a guiding signal to facilitate accurate estimation of $c_t^{\mathrm{prefix}}$.
• Figure 3: Layer-wise analysis of step-level hallucination probing. F1 score (top) and AUROC (bottom) are reported across transformer layers for three base models.
• Figure 4: Step-level probing performance across different CoT positions on LLaMA-3.1-8B. We compare five probe variants with different representation aggregation strategies, where Step Time Exponential is our final choice. AUC is threshold-free, while ACC and F1 use a fixed threshold of $0.5$.
• Figure 5: Radar visualization of eight dynamic metrics for prefix-level hallucination evaluation, all normalized to a $[0,100]$ scale. Left: LLaMA; Right: Qwen. Detailed definitions of all metrics are provided in Appendix \ref{['appx:dynamic_metrics']}.