A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputs

TL;DR

The paper tackles the persistent problem of hallucinations in LLM outputs by introducing pre-trained Uncertainty Quantification (UQ) heads, supervised auxiliary modules that leverage a Transformer backbone and attention-based signals to estimate the uncertainty of atomic claims without altering the LLMs themselves. The authors show that these UQ heads achieve state-of-the-art claim-level hallucination detection, generalize across domains and languages, and remain computationally lightweight. They propose a data-generation pipeline to create large-scale, claim-level annotations and demonstrate strong empirical gains against both unsupervised and supervised baselines across multiple LLMs and languages. Finally, they release a suite of pre-trained UQ heads for popular models, underscoring the practical utility of plug-and-play uncertainty detection in real-world deployments.

Abstract

Large Language Models (LLMs) have the tendency to hallucinate, i.e., to sporadically generate false or fabricated information. This presents a major challenge, as hallucinations often appear highly convincing and users generally lack the tools to detect them. Uncertainty quantification (UQ) provides a framework for assessing the reliability of model outputs, aiding in the identification of potential hallucinations. In this work, we introduce pre-trained UQ heads: supervised auxiliary modules for LLMs that substantially enhance their ability to capture uncertainty compared to unsupervised UQ methods. Their strong performance stems from the powerful Transformer architecture in their design and informative features derived from LLM attention maps. Experimental evaluation shows that these heads are highly robust and achieve state-of-the-art performance in claim-level hallucination detection across both in-domain and out-of-domain prompts. Moreover, these modules demonstrate strong generalization to languages they were not explicitly trained on. We pre-train a collection of UQ heads for popular LLM series, including Mistral, Llama, and Gemma 2. We publicly release both the code and the pre-trained heads.

Figures (6)

• Figure 1: The architecture of uncertainty quantification heads. The example represents a text generated using an LLM, containing the hallucination 20 Grammy Awards highlighted in red.
• Figure 2: (a) The distribution of correlations between attention on the $i-1$-th token and presence of the $i$-th token in hallucinated claim. (b) The maximum absolute correlation across heads and layers for the same phenomenon. All scores were computed using the Mistral model and the biographies dataset.
• Figure 3: Code example for using uncertainty heads.
• Figure 4: The training data generation pipeline.
• Figure 5: PR-AUC for different attention window sizes using UHead for Mistral 7B Instruct v0.2 model.