Probabilistic Guarantees for Reducing Contextual Hallucinations in LLMs

TL;DR

This work addresses contextual hallucinations in fixed-input LLM workflows by proposing a model-agnostic framework that combines independent repetitions with an LLM-based judge. By treating each run as an independent draw from a fixed task distribution, the probability that all runs fail decays exponentially as $(1-p)^N$, where $p$ is the per-run correctness. When a judge is introduced, the end-to-end reliability depends on the judge’s true- and false-positive rates $q^{++}$ and $q^{-+}$, with the pipeline failure probability bounded by $(1-(p q^{++}+(1-p) q^{-+}))^N$; ensemble judging with $K$ judges further drives hallucination-selection down via a Hoeffding-bound guarantee $Q^{-+}(K)\le e^{-2K( frac12-q^{-+})^2}$. Experiments on controlled extraction tasks with synthetic noisy judges confirm the theory: both the existence and identification guarantees hold, with error rates decaying exponentially in $N$ and $K$, respectively. The approach offers a lightweight, modular path to extremely reliable fixed-input LLM workflows without changing model weights, decoding strategies, or prompt engineering, and suggests practical directions for extending to broader task classes and robustness analyses.

Abstract

Large language models (LLMs) frequently produce contextual hallucinations, where generated content contradicts or ignores information explicitly stated in the prompt. Such errors are particularly problematic in deterministic automation workflows, where inputs are fixed and correctness is unambiguous. We introduce a simple and model-agnostic framework that provides explicit probabilistic guarantees for reducing hallucinations in this setting. We formalize the notion of a specific task, defined by a fixed input and a deterministic correctness criterion, and show that issuing the same prompt in independent context windows yields an exponential reduction in the probability that all model outputs are incorrect. To identify a correct answer among repeated runs, we incorporate an LLM-as-a-judge and prove that the probability that the judged pipeline fails decays at a rate determined by the judge's true- and false-positive probabilities. When the judge is imperfect, we strengthen it through majority vote over independent judge calls, obtaining ensemble-level error rates that decrease exponentially in the number of votes. This yields an explicit bound on the probability that the pipeline selects a hallucinated answer. Experiments on controlled extraction tasks with synthetic noisy judges match these predictions exactly: pipeline failure decreases exponentially with the number of repetitions, and hallucination-selection decreases exponentially with the number of judges in the ensemble. Together, these results provide a lightweight, modular, and theoretically grounded method for driving hallucination probabilities arbitrarily low in fixed-input LLM workflows-without modifying model weights, decoding strategies, or prompt engineering.

Figures (2)

• Figure 1: The repetition-judge pipeline gives probabilistic guarantees for reducing contextual hallucinations. Left: For a given specific task, the likelihood that all answers are hallucinated is reduced by repeating the task $N$ times and judging the output $X_n$ of each for correctness. The pipeline fails ("all answers are judged as incorrect") exponentially decreasing with $N$. Right: From the judged-correct answers, a true correct answer can be found at guaranteed rates. If the judge is perfect ($q^{++}=1,\, q^{-+}=0$), any $X_n$ with $Y_n=1$ is a correct answer. If the judge is noisy but better than random ($q^{-+}<0.5$), an ensemble judge selects a hallucination exponentially decreasing with the number of judges $K$.
• Figure 2: Empirical performance of the repetition--judge pipeline and ensemble judge, estimated over $10,000$ pipeline runs. We present three example tasks with different $p$ and use an imperfect judge ($q^{++}=0.75,\,q^{-+}=0.25$). a--c) Pipeline failure rate ("all answers are judged as incorrect") decreases exponentially with the number of repetitions $N$. Theorem \ref{['thm:pipeline-failure']} predicts the observed decay, indicated by the dashed line. d--f) Hallucination-selection rate ("the ensemble judge selects a hallucinated answer") decreases exponentially with ensemble size $K$. Theorem \ref{['thm:ensemble-hallucination']} predicts the observed decay, indicated by the dashed line. All error bars indicate Wilson confidence intervals. Together, these demonstrate that the repetition-judge pipeline lays the foundation for reducing contextual hallucinations to desired rates, even for imperfect judges.

Theorems & Definitions (6)

• Theorem 2.1: Repetition Lemma
• proof
• Theorem 3.1: Reliability of the repetition-judge pipeline
• proof
• Theorem 3.2: Hallucination-selection with an ensemble judge
• proof