Harnessing Reasoning Trajectories for Hallucination Detection via Answer-agreement Representation Shaping

TL;DR

Answer-agreement Representation Shaping (ARS), which learns detection-friendly trace-conditioned representations by explicitly encoding answer stability, and consistently improves detection and achieves substantial gains over strong baselines.

Abstract

Large reasoning models (LRMs) often generate long, seemingly coherent reasoning traces yet still produce incorrect answers, making hallucination detection challenging. Although trajectories contain useful signals, directly using trace text or vanilla hidden states for detection is brittle: traces vary in form and detectors can overfit to superficial patterns rather than answer validity. We introduce Answer-agreement Representation Shaping (ARS), which learns detection-friendly trace-conditioned representations by explicitly encoding answer stability. ARS generates counterfactual answers through small latent interventions, specifically, perturbing the trace-boundary embedding, and labels each perturbation by whether the resulting answer agrees with the original. It then learns representations that bring answer-agreeing states together and separate answer-disagreeing ones, exposing latent instability indicative of hallucination risk. The shaped embeddings are plug-and-play with existing embedding-based detectors and require no human annotations during training. Experiments demonstrate that ARS consistently improves detection and achieves substantial gains over strong baselines.

Figures (14)

• Figure 1: Effect of reasoning trajectories on hallucination detection in LRMs. We compare detection performance for the same LRM (Qwen3-8B yang2025qwen3technicalreport) with and without an explicit reasoning trajectory, using representations extracted from each layer for the same answers. Consistent with our hypothesis, reasoning traces can sometimes obscure answer-level hallucination signals. The dataset is TruthfulQA lin2022truthfulqa.
• Figure 2: Overview of ARS framework for hallucination detection in LRMs. ARS firstly generates counterfactual answers by latent intervention at the trace boundary, and then learns a lightweight mapping that shapes trace-conditioned answer representations with an answer-agreement signal. This can make truthful vs. hallucinated outputs more separable for downstream embedding-based detectors.
• Figure 3: (left) Generalization across datasets, where “(s)” denotes the source data and “(t)” denotes the target data. (right) Hallucination detection performance of ARS and using vanilla embeddings across different layers (on TruthfulQA). Model used is Qwen3-8B for both (left) and (right).
• Figure 4: (a) Hallucination detection scores of our ARS and using vanilla LRM embeddings (Reasoning trace is omitted for easier presentation. Larger score denotes more truthfulness). (b) Counterfactual answers generated for the examples in (a). (c) Embedding visualization comparing vanilla and ARS-shaped representation. The model is Qwen3-8B and we utilize questions in TruthfulQA.
• Figure 5: (a) Effect of intervention position, (b) effect of intervention strength $\sigma$, (c) effect of output dimension $k$ for the trace-conditioned answer representations, and (d) effect of number of counterfactual answers $M$. All results are reported on TruthfulQA using Qwen3-8B. The downstream detector is probing.

Theorems & Definitions (9)

• Definition 4.1: Answer stability score
• Proposition 4.2
• Definition J.1: Conditional and marginal agreement separation
• Lemma J.2: Existence of a stability surrogate from $\bm{z}$
• proof
• Lemma J.4: Plug-in thresholding with an approximate stability surrogate
• proof
• Proposition J.6: Agreement separation $\Rightarrow$ bounded error of supervised probing
• proof