Two Pathways to Truthfulness: On the Intrinsic Encoding of LLM Hallucinations

TL;DR

This work investigates why LLMs encode truthfulness internally by identifying two distinct information pathways: a Q-Anchored pathway that relies on question-to-answer information flow and an A-Anchored pathway that uses self-contained evidence from the generated answer. Through attention knockout and token patching experiments across 12 models and four QA datasets, the authors show that Q-Anchored signals depend on question-derived cues while A-Anchored signals persist when such cues are removed. They reveal two key properties: association with knowledge boundaries (Q-Anchored for well-known facts; A-Anchored for long-tail data) and intrinsic self-awareness of pathway distinctions (internal states can predict the active pathway). Building on these insights, they propose Mixture-of-Probes and Pathway Reweighting as pathway-aware detection methods, achieving up to ~10% AUC gains and offering practical routes to more reliable and self-aware generative systems.

Abstract

Despite their impressive capabilities, large language models (LLMs) frequently generate hallucinations. Previous work shows that their internal states encode rich signals of truthfulness, yet the origins and mechanisms of these signals remain unclear. In this paper, we demonstrate that truthfulness cues arise from two distinct information pathways: (1) a Question-Anchored pathway that depends on question-answer information flow, and (2) an Answer-Anchored pathway that derives self-contained evidence from the generated answer itself. First, we validate and disentangle these pathways through attention knockout and token patching. Afterwards, we uncover notable and intriguing properties of these two mechanisms. Further experiments reveal that (1) the two mechanisms are closely associated with LLM knowledge boundaries; and (2) internal representations are aware of their distinctions. Finally, building on these insightful findings, two applications are proposed to enhance hallucination detection performance. Overall, our work provides new insight into how LLMs internally encode truthfulness, offering directions for more reliable and self-aware generative systems.

Figures (35)

• Figure 1: Kernel density estimates of saliency‐score distributions for critical question-to-answer information flows. The bimodal pattern suggests two distinct information mechanisms.
• Figure 2: $\Delta \mathrm{P}$ under attention knockout. The layer axis indicates the Transformer layer on which the probe is trained. Shaded regions indicate 95% confidence intervals. Full results in Appendix \ref{['sec:appendix_attn_knockout']}.
• Figure 3: Prediction flip rate under token patching. Q-Anchored samples demonstrate significantly higher sensitivity than the counterparts when hallucinatory cues are injected into exact questions. Full results in Appendix \ref{['sec:appendix_token_patching']}.
• Figure 4: $-\Delta \mathrm{P}$ with only the LLM-generated answer. Q-Anchored instances exhibit substantial shifts, whereas A-Anchored instances remain stable, confirming that A-Anchored truthfulness encoding relies on information in the LLM-generated answer itself. Full results in Appendix \ref{['sec:appendix_answer_only_input']}.
• Figure 5: Comparisons of answer accuracy between pathways. Q-Anchored samples show higher accuracy than A-Anchored ones, highlighting the association between truthfulness encoding and LLM knowledge boundaries. Full results in Appendix \ref{['sec:appendix_answer_accuracy']} and \ref{['sec:appendix_idk']}.