HalluShift: Measuring Distribution Shifts towards Hallucination Detection in LLMs

TL;DR

HalluShift addresses the hallucination problem in LLMs by detecting factual inconsistencies through internal-state dynamics and token-probability signals, rather than relying solely on external evaluators. It combines distribution-shift features captured via Wasserstein distances and cosine similarities across model layers with probabilistic token-level features in a two-layer metric-learning MLP to produce a hallucination score. The approach achieves state-of-the-art or near-state-of-the-art performance across multiple benchmarks and models, while maintaining a favorable computational footprint of $O(m^2)$. The findings suggest hallucinations leave robust fingerprints in the way information propagates through the LLM’s layers, and HalluShift provides a practical, open-box method for detection with cross-domain robustness and potential for integration into mitigation strategies.

Abstract

Large Language Models (LLMs) have recently garnered widespread attention due to their adeptness at generating innovative responses to the given prompts across a multitude of domains. However, LLMs often suffer from the inherent limitation of hallucinations and generate incorrect information while maintaining well-structured and coherent responses. In this work, we hypothesize that hallucinations stem from the internal dynamics of LLMs. Our observations indicate that, during passage generation, LLMs tend to deviate from factual accuracy in subtle parts of responses, eventually shifting toward misinformation. This phenomenon bears a resemblance to human cognition, where individuals may hallucinate while maintaining logical coherence, embedding uncertainty within minor segments of their speech. To investigate this further, we introduce an innovative approach, HalluShift, designed to analyze the distribution shifts in the internal state space and token probabilities of the LLM-generated responses. Our method attains superior performance compared to existing baselines across various benchmark datasets. Our codebase is available at https://github.com/sharanya-dasgupta001/hallushift.

Figures (5)

• Figure 1: Comparison of AUC-ROC (%) across different QA datasets for hallucination detection using various methods, evaluated on OPT6.7B (a) and LLaMA-2-7B (b) models.
• Figure 2: Illustration of our proposed method, HalluShift. Firstly, we extract the features on a fixed window $\mathbf{r}$ over the internal layers of LLM to analyze generations, considering distribution shifts $W(\mu_l^j, \mu_{l+r}^j)$ and cosine similarities $C(z_l^j, z_{l+r}^j)$ across model hidden states and attentions. Given generative features from models, distribution, and cosine can have similar drifts when hallucination takes place (noted in the first generation from green to red). Secondly, it uses probabilistic features $(mtp, Mps, \cdots , Mg)$ constructed from token-level uncertainty $(p_1, p_2, ... )$ to estimate the membership (factual vs. inaccurate) through a membership function that assigns a hallucination score to each generation $y$.
• Figure 3: Examples present a comparative analysis of HalluShift's hallucination score against human evaluation metrics using diverse TruthfulQA examples. By comparing HalluShift's assessment with expert human judgment, we demonstrate the reliability and precision of our hallucination detection approach across multiple generated textual responses to identical input prompts.
• Figure 4: We present a dual-perspective analysis to evaluate HalluShift. Left: Feature importance analysis via feature perturbation reveals hallucination detection model predictive sensitivity through controlled Gaussian noise introduction and deviation measurement across individual features. Right: Generalization across four QA datasets, where training datasets are mapped along the x-axis and testing datasets are mapped along the y-axis.
• Figure 5: Architecture of membership estimation function : Metric Learning with Distribution Shift, Similarity and Probabilistic Features for Hallucination Detection.