LUMINA: Detecting Hallucinations in RAG System with Context-Knowledge Signals

TL;DR

Lumina addresses hallucinations in Retrieval-Augmented Generation by jointly measuring how much external context and internal knowledge influence each generation step. It quantifies external context utilization with a distributional distance (Maximum Mean Discrepancy) between token-embedding distributions conditioned on real vs. random documents, and internal knowledge utilization with an information processing rate derived from layerwise predictions via LogitLens. A statistical validation framework links these signals to verifiable implications, and experiments across four open LLMs on RAGTruth and HalluRAG show state-of-the-art detection performance, robustness to retrieval noise, and competitiveness with supervised baselines while remaining unsupervised. Practically, Lumina offers a scalable, hyperparameter-efficient approach to improve the reliability of RAG systems and provides interpretable insights into when and why hallucinations occur.

Abstract

Retrieval-Augmented Generation (RAG) aims to mitigate hallucinations in large language models (LLMs) by grounding responses in retrieved documents. Yet, RAG-based LLMs still hallucinate even when provided with correct and sufficient context. A growing line of work suggests that this stems from an imbalance between how models use external context and their internal knowledge, and several approaches have attempted to quantify these signals for hallucination detection. However, existing methods require extensive hyperparameter tuning, limiting their generalizability. We propose LUMINA, a novel framework that detects hallucinations in RAG systems through context-knowledge signals: external context utilization is quantified via distributional distance, while internal knowledge utilization is measured by tracking how predicted tokens evolve across transformer layers. We further introduce a framework for statistically validating these measurements. Experiments on common RAG hallucination benchmarks and four open-source LLMs show that LUMINA achieves consistently high AUROC and AUPRC scores, outperforming prior utilization-based methods by up to +13% AUROC on HalluRAG. Moreover, LUMINA remains robust under relaxed assumptions about retrieval quality and model matching, offering both effectiveness and practicality.

Figures (6)

• Figure 1: The overview of Lumina. For external context utilization, we propose to measure the maximum mean discrepancy between two next token probability distributions conditioned on different documents. For internal knowledge utilization, we introduce the idea of information processing rate by looking at the ratio of the most probable output token's probability across transformer layers and use it to determine the amount of utilized internal knowledge when generating the next token.
• Figure 2: Noises in context do not largely degrade the performance of Lumina. We add $0\sim30\%$ noises to the retrieved documents and random documents and evaluate the hallucination detection performance. The experiment is conducted on the RAGTruth dataset.
• Figure 3: The "same LLM" setting is not essential for Lumina to achieve the optimal performance. On the RAGTruth dataset, for each set of responses generated by the same LLM, we apply Lumina with a different base LLM to detect hallucination. Bars in more saturated shades indicate settings where the same LLM is used for both generation and detection.
• Figure 4: Combining scores of external context and internal knowledge boosts the hallucination detection performance. Left: 2D kernel density estimation (KDE) of the distribution of external context score and internal knowledge score of Llama2-13B responses on the RAGTruth dataset. Right: Hallucination detection performance with external/internal score only, as well as the performance of their combination.
• Figure 5: MMD with cosine kernel performs similarly or better than with RBF kernel.

Theorems & Definitions (4)

• Conjecture 1: External context vs. internal knowledge utilization
• Definition 2.1: Hallucination in an RAG system
• Definition 3.1: Maximum Mean Discrepancy Gretton_Borgwardt_Rasch_Schölkopf_Smola_2012
• Definition 3.2: Information processing rate