Relational Linearity is a Predictor of Hallucinations

TL;DR

This work investigates why LLMs hallucinate when asked about unknown subjects by proposing that the linearity of relation representations predicts hallucination likelihood. They introduce Synt-Hal, a synthetic dataset with six relations (covering both linear and nonlinear structures), and measure relational linearity with the $Δ\cos$ metric derived from Linear Relational Embeddings, using an LLM-as-a-judge to label outputs as hallucinations or refusals. Across four instruction-tuned models, they find a strong positive correlation between $Δ\cos$ and hallucination rate on the synthetic data ($r$ between approximately $0.78$ and $0.82$), while natural triples show the opposite trend, suggesting different dynamics when prior knowledge is present. The results imply that abstract, relation-wide representations of linear triples hinder knowledge self-assessment, pointing to directions for augmenting linear-relation representations to improve truthfulness and reduce hallucinations in LLMs.

Abstract

Hallucination is a central failure mode in large language models (LLMs). We focus on hallucinations of answers to questions like: "Which instrument did Glenn Gould play?", but we ask these questions for synthetic entities that are unknown to the model. Surprisingly, we find that medium-size models like Gemma-7B-IT frequently hallucinate, i.e., they have difficulty recognizing that the hallucinated fact is not part of their knowledge. We hypothesize that an important factor in causing these hallucinations is the linearity of the relation: linear relations tend to be stored more abstractly, making it difficult for the LLM to assess its knowledge; the facts of nonlinear relations tend to be stored more directly, making knowledge assessment easier. To investigate this hypothesis, we create SyntHal, a dataset of 6000 synthetic entities for six relations. In our experiments with four models, we determine, for each relation, the hallucination rate on SyntHal and also measure its linearity, using $Δ\cos$. We find a strong correlation ($r \in [.78,.82]$) between relational linearity and hallucination rate, providing evidence for our hypothesis that the underlying storage of triples of a relation is a factor in how well a model can self-assess its knowledge. This finding has implications for how to manage hallucination behavior and suggests new research directions for improving the representation of factual knowledge in LLMs.

Figures (4)

• Figure 1: Hallucination rate (# hallucinations divided by (# hallucinations plus # refusals)) vs. relational linearity (measured by $\Delta\cos$) on Synt-Hal (i.e., synthetic data that the model does not know the answer for). Each point is a relation. Titles report Pearson's $r$. Higher $\Delta\cos$ is associated with higher hallucination rates across all four models. The correlations are significant at $p<.1$ (two-sided t-test), with three $p$ values slightly above the $p<.05$ threshold: .0662, .0567, .0511.
• Figure 2: Hallucination rate (# hallucinations divided by (# hallucinations plus # correct)) vs. relational linearity (measured by $\Delta\cos$) on LRE. Each point is a relation. Titles report Pearson's $r$. Higher $\Delta\cos$ is associated with lower hallucination rate across all four models. The correlations are significant at $p<.05$.
• Figure 3: Two model generations and LLM-as-a-judge annotations. Judge rationale omitted.
• Figure 4: Per-relation behavioral outcomes under the standardized inference interface: hallucination (value-providing) vs. refusal rates with 95% Wilson score CIs, computed over $N{=}1000$ synthetic prompts per relation. This figure is a visual counterpart to Table \ref{['tab:hall_rate_ci']}.