Sample Transform Cost-Based Training-Free Hallucination Detector for Large Language Models

Abstract

Hallucinations in large language models (LLMs) remain a central obstacle to trustworthy deployment, motivating detectors that are accurate, lightweight, and broadly applicable. Since an LLM with a prompt defines a conditional distribution, we argue that the complexity of the distribution is an indicator of hallucination. However, the density of the distribution is unknown and the samples (i.e., responses generated for the prompt) are discrete distributions, which leads to a significant challenge in quantifying the complexity of the distribution. We propose to compute the optimal-transport distances between the sets of token embeddings of pairwise samples, which yields a Wasserstein distance matrix measuring the costs of transforming between the samples. This Wasserstein distance matrix provides a means to quantify the complexity of the distribution defined by the LLM with the prompt. Based on the Wasserstein distance matrix, we derive two complementary signals: AvgWD, measuring the average cost, and EigenWD, measuring the cost complexity. This leads to a training-free detector for hallucinations in LLMs. We further extend the framework to black-box LLMs via teacher forcing with an accessible teacher model. Experiments show that AvgWD and EigenWD are competitive with strong uncertainty baselines and provide complementary behavior across models and datasets, highlighting distribution complexity as an effective signal for LLM truthfulness.

Figures (6)

• Figure 1: Overview of distribution-consistency detection. For a given prompt, we sample multiple responses, extract generated-token hidden states, compute pairwise Wasserstein distances between responses, and summarize the resulting structure using AvgWD (average pairwise transform cost) and EigenWD (spectral complexity of the induced cost matrix).
• Figure 2: Heatmaps of sample-to-sample OT costs on CoQA. For a prompt, $D_{ij}=W_2(\mu_i,\mu_j)$ is the Wasserstein distance between the $i$-th and $j$-th sampled responses (diagonal is zero); darker cells indicate larger transform costs. Hallucinated cases often exhibit larger average costs and/or more fragmented block structure, motivating AvgWD (magnitude) and EigenWD (structure).
• Figure 3: Heatmaps of sample-to-sample OT costs on SQuAD. For a prompt, $D_{ij}=W_2(\mu_i,\mu_j)$ is the Wasserstein distance between the $i$-th and $j$-th sampled responses (diagonal is zero); darker cells indicate larger transform costs. Hallucinated cases often exhibit larger average costs and/or more fragmented block structure, motivating AvgWD (magnitude) and EigenWD (structure).
• Figure 4: Cost-graph visualization from sample transform costs. Each node is a sampled response for a prompt; node positions come from a 2D embedding of the pairwise Wasserstein matrix $D$ (precomputed distances), and edges connect nearest-neighbor pairs under $D$ (solid: hallucinated cases; dashed: non-hallucinated). Marker shapes (A1--A3) denote different representative prompts within each group.
• Figure 5: Ablation results on Llama-3.1-8B. Left: AUROC versus the number of sampled responses $K$. Right: AUROC versus temperature $\tau$. Unless otherwise stated, we keep decoding settings (including top-$k$ and top-$\rho$) fixed across runs.

Theorems & Definitions (9)

• Lemma 1: Two-sided Wasserstein stability
• Lemma 2: Token-level perturbation bound
• Theorem 1: AvgWD is Lipschitz under token-level perturbations
• Corollary 1
• proof
• proof
• proof
• Lemma 3: EigenWD is locally Lipschitz in $\mathbf{D}$
• proof