Semantic Uncertainty Quantification of Hallucinations in LLMs: A Quantum Tensor Network Based Method

TL;DR

The paper tackles the challenge of hallucinations in LLMs by introducing a physics-inspired uncertainty framework that treats token-sequence probabilities as a data wave function within a quantum tensor network (QTN). It combines semantic clustering via bidirectional entailment with a kernel-based Rényi entropy (SE_R) and a perturbation-based UQ pipeline to calibrate TS probabilities through a maximum-entropy objective, resulting in an uncertainty-adjusted probability $p_s^{(r)^*}$. The method is validated across TriviaQA, NQ-Open, SVAMP, and SQuAD on diverse models, including quantized variants, with 116 experiments showing robust improvements in AUROC and AURAC without supervised fine-tuning. The approach enables principled, scalable hallucination detection and even informed answer selection under resource constraints, highlighting its practical potential for safer LLM deployment. Limitations include reliance on external entailment predictors and access to token-level probabilities, suggesting avenues for future work with larger models and black-box settings, while maintaining the core idea of uncertainty-aware semantic evaluation.

Abstract

Large language models (LLMs) exhibit strong generative capabilities but remain vulnerable to confabulations, fluent yet unreliable outputs that vary arbitrarily even under identical prompts. Leveraging a quantum tensor network based pipeline, we propose a quantum physics inspired uncertainty quantification framework that accounts for aleatoric uncertainty in token sequence probability for semantic equivalence based clustering of LLM generations. This offers a principled and interpretable scheme for hallucination detection. We further introduce an entropy maximization strategy that prioritizes high certainty, semantically coherent outputs and highlights entropy regions where LLM decisions are likely to be unreliable, offering practical guidelines for when human oversight is warranted. We evaluate the robustness of our scheme under different generation lengths and quantization levels, dimensions overlooked in prior studies, demonstrating that our approach remains reliable even in resource constrained deployments. A total of 116 experiments on TriviaQA, NQ, SVAMP, and SQuAD across multiple architectures including Mistral-7B, Mistral-7B-instruct, Falcon-rw-1b, LLaMA-3.2-1b, LLaMA-2-13b-chat, LLaMA-2-7b-chat, LLaMA-2-13b, and LLaMA-2-7b show consistent improvements in AUROC and AURAC over state of the art baselines.

Figures (24)

• Figure 1: Overview of our hallucination detection pipeline. Sequences are clustered via directional entailment, and UQ obtained through QTN is used for entropy maximization to enable reliable LLM hallucination detection.
• Figure 2: Pairwise win rate matrices across SOTA hallucination detection methods on diverse datasets and LLM models. This summarizes 116 experimental scenarios. Each cell indicates the probability that the row model outperforms the column model. Semantic Rényi entropy with UQ maximization consistently outperforms baselines, even surpassing methods reliant on supervised learning.
• Figure 3: Evaluation of SOTA hallucination detection methods across different quantization levels. This summarizes 116 experimental scenarios. Each subfigure shows pairwise win rate matrices under a different quantization precision (16-bit, 8-bit, 4-bit). SRE-UQ maximization consistently outperforms baselines across both AUROC and AURAC, demonstrating robustness in hallucination detection against reduced precision.
• Figure 4: Signed normalized difference (ND) in entropy across old entropy bins for Llama-2 variants under different quantization settings (16-bit, 8-bit, 4-bit). Bars represent the mean signed ND with error bars indicating standard error. Notably, all models exhibit the largest variability in the 0.25–0.50 entropy range, suggesting this is a high-risk region where models frequently oscillate between multiple confident yet semantically divergent answers.
• Figure 5: Overview of the QTN-based UQ pipeline used in our work. The procedure begins with computing the quantum correlation matrix (QCM) from TS probabilities and extracting its null-space vectors. These are used to construct the local Hamiltonian $\widehat{\ul{H}}$, whose eigen-modes provide the foundation for spectral and perturbation analysis. First-order perturbation corrections yield uncertainty feature vectors, which are organized into a tensor network representation. Finally, sampling across modes produces uncertainty bounds on probability amplitudes, enabling uncertainty quantification of semantic entropy.

Theorems & Definitions (4)

• Lemma 1
• Lemma 2
• Corollary 3
• Corollary 4