Hallucination as a Computational Boundary: A Hierarchy of Inevitability and the Oracle Escape

TL;DR

This work formalizes hallucination as an intrinsic limit when LLMs are modeled as probabilistic Turing machines, introducing a three-boundary hierarchy (Diagonalization, Uncomputability, Information-Theoretic) and two escape strategies: external Oracle augmentation (RAG) and internal continual learning (CL). It develops a neuro-game-theoretic, CLS-inspired framework for adaptive escape, proves amortized cost advantages for continual learning, and introduces Computational Class Alignment (CCA) as a safety principle. The experimental validation with a RAG-CL hybrid demonstrates robust accuracy with cost benefits, supporting dynamic alignment of task complexity to the agent's computational class. Overall, the paper shifts safety design from single-model fixes to ecosystemwide, boundary-aware strategies that adaptively elevate internal capabilities.

Abstract

The illusion phenomenon of large language models (LLMs) is the core obstacle to their reliable deployment. This article formalizes the large language model as a probabilistic Turing machine by constructing a "computational necessity hierarchy", and for the first time proves the illusions are inevitable on diagonalization, incomputability, and information theory boundaries supported by the new "learner pump lemma". However, we propose two "escape routes": one is to model Retrieval Enhanced Generations (RAGs) as oracle machines, proving their absolute escape through "computational jumps", providing the first formal theory for the effectiveness of RAGs; The second is to formalize continuous learning as an "internalized oracle" mechanism and implement this path through a novel neural game theory framework. Finally, this article proposes a feasible new principle for artificial intelligence security - Computational Class Alignment (CCA), which requires strict matching between task complexity and the actual computing power of the system, providing theoretical support for the secure application of artificial intelligence.

Figures (6)

• Figure 1: Flowchart for the Halting Problem decider, $\mathcal{M}'$. Its existence, enabled by a hypothetical low-hallucination PLM, contradicts Turing's proof, thus proving the PLM cannot exist.
• Figure 2: The information bottleneck in the Pumping Lemma. A high-complexity truth, containing an incompressible random patch $z$, is too large to fit through the model's finite capacity $\mathrm{K}(h)$, leading to information loss and large hallucination.
• Figure 3: The adversarial paradox for a standard PLM. A self-referential input $s^*$ forces the model $h$ to generate a prediction $y_h^*$. The oracle $\mathcal{O}$ is then defined to explicitly contradict this prediction, guaranteeing hallucination.
• Figure 4: Operational flowchart of the oracle-augmented PLM, $h^{\mathcal{O}}$. The model bypasses internal computation and directly queries the oracle, ensuring perfect alignment.
• Figure 5: Attention Shift Analysis for the RAG-CL Hybrid. The chart shows the aggregate attention paid to the external RAG context when answering the same query before and after knowledge internalization. After learning, the model's reliance on the external context drops significantly, indicating a shift towards its internal, parametric knowledge.

Theorems & Definitions (14)

• Definition 2.1: Probabilistic Language Model (PLM)
• Definition 2.2: Refined Hallucination Metrics
• Definition 2.3: Oracle Machine and Kolmogorov Complexity
• Theorem 2.4
• proof
• Theorem 2.5
• proof
• Lemma 2.6: A Pumping Lemma for Learners
• proof
• Theorem 3.1: The Oracle Escape Theorem