Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hallucinations Live in Variance

Aaron R. Flouro, Shawn P. Chadwick

TL;DR

The work identifies reliability under semantic perturbation as a missing axis in standard benchmarks and defines hallucinations as variance-driven instability rather than incorrectness alone. It formalizes Semantic Stability (SS) and Paraphrase Consistency (PC@k) under deterministic decoding, showing SS exposes variance-driven unreliability independent of ground-truth accuracy. Through a SparseKD-based phase diagram, the paper demonstrates a non-monotonic stability peak at moderate sparsity (e.g., 32%), where variance reduction can improve stability without excessive bias. Importantly, SS is diagnostic and benchmark-agnostic, but must be paired with accuracy, as higher SS can coincide with biased, incorrect answers; the approach provides actionable guidance for reliability-focused deployment and evaluation in regulated settings.

Abstract

Benchmarks measure whether a model is correct. They do not measure whether a model is reliable. This distinction is largely academic for single-shot inference, but becomes critical for agentic AI systems, where a single rephrased prompt can trigger cascading failures in multi-step execution. Yet this form of instability is not captured by existing evaluations. Hallucinations live in variance: they arise when semantically equivalent prompts activate inconsistent internal pathways, producing divergent outputs. Consistent but incorrect outputs reflect bias or missing knowledge; confident guessing reflects calibration failure. Neither constitutes hallucination under this definition. When error is variance-dominated, reducing redundant pathways improves reliability without adding knowledge. We formalize this through Semantic Stability (SS), measured via Paraphrase Consistency (PC@k): generate k paraphrases, greedy decode each, compute mode agreement. SS is a diagnostic for variance-driven unreliability, not a method for improving correctness. We show that a dense Qwen3-0.6B agrees with itself only 23.8% of the time; at 32% sparsity, agreement jumps to 55.9%. A phase diagram reveals the sweet spot where variance reduction outpaces bias accumulation, and regimes where stability collapses onto wrong answers.

Hallucinations Live in Variance

TL;DR

The work identifies reliability under semantic perturbation as a missing axis in standard benchmarks and defines hallucinations as variance-driven instability rather than incorrectness alone. It formalizes Semantic Stability (SS) and Paraphrase Consistency (PC@k) under deterministic decoding, showing SS exposes variance-driven unreliability independent of ground-truth accuracy. Through a SparseKD-based phase diagram, the paper demonstrates a non-monotonic stability peak at moderate sparsity (e.g., 32%), where variance reduction can improve stability without excessive bias. Importantly, SS is diagnostic and benchmark-agnostic, but must be paired with accuracy, as higher SS can coincide with biased, incorrect answers; the approach provides actionable guidance for reliability-focused deployment and evaluation in regulated settings.

Abstract

Benchmarks measure whether a model is correct. They do not measure whether a model is reliable. This distinction is largely academic for single-shot inference, but becomes critical for agentic AI systems, where a single rephrased prompt can trigger cascading failures in multi-step execution. Yet this form of instability is not captured by existing evaluations. Hallucinations live in variance: they arise when semantically equivalent prompts activate inconsistent internal pathways, producing divergent outputs. Consistent but incorrect outputs reflect bias or missing knowledge; confident guessing reflects calibration failure. Neither constitutes hallucination under this definition. When error is variance-dominated, reducing redundant pathways improves reliability without adding knowledge. We formalize this through Semantic Stability (SS), measured via Paraphrase Consistency (PC@k): generate k paraphrases, greedy decode each, compute mode agreement. SS is a diagnostic for variance-driven unreliability, not a method for improving correctness. We show that a dense Qwen3-0.6B agrees with itself only 23.8% of the time; at 32% sparsity, agreement jumps to 55.9%. A phase diagram reveals the sweet spot where variance reduction outpaces bias accumulation, and regimes where stability collapses onto wrong answers.
Paper Structure (10 sections, 7 equations, 2 figures, 2 tables)

This paper contains 10 sections, 7 equations, 2 figures, 2 tables.

Table of Contents

  1. The Missing Axis
  2. A Metric for Reliability: Semantic Stability
  3. Why Existing Metrics Miss This
  4. Related Work and Limitations
  5. Variance as the Mechanism
  6. The Phase Diagram: Stability vs Sparsity
  7. Adding SS to Any Benchmark
  8. Stable Does Not Mean Correct
  9. Implications
  10. Conclusion

Figures (2)

  • Figure 1: Compression as variance reduction. (A) Teacher has high variance around mean $\mu$. (B) One-shot pruning shifts the mean and distorts the distribution. (C) Staged pruning preserves $\mu$ while reducing variance.
  • Figure 2: The SparseKD stability phase diagram. Stability peaks at 32% sparsity (R4). Blue: variance-dominated regime where compression tightens distributions. Green: optimal SparseKD zone. Red: bias-dominated regime where over-pruning collapses onto wrong answers.

Theorems & Definitions (1)

  • Definition 2.1: Paraphrase Consistency and Semantic Stability