From "Hallucination" to "Suture": Insights from Language Philosophy to Enhance Large Language Models

TL;DR

This paper addresses hallucinations in large language models by importing Lacanian language theory and proposing the Anchor-RAG framework to ground predictions. It formalizes the approach with a RAG-based grounding mechanism that identifies quilting points (anchors) via masking and top-$k$ analysis, retrieves context for these anchors, and uses controlled generation to reduce unsupported content. Theoretical foundations are articulated through $P(Y \mid X) = \sum_{D \in \mathcal{D}} P(Y \mid X, D) \cdot P(D \mid X)$ and anchor-conditioned generation $P(Y \mid X, A) = \prod_{t=1}^{T} P(y_t \mid Y_{<t}, X, A)$, with entropy and cosine similarity guiding anchor selection and retrieval. While empirical results are pending, the work argues for improved grounding, interpretability, and a shift away from a purely trial-and-error paradigm toward linguistically informed AI design.

Abstract

This paper explores hallucination phenomena in large language models (LLMs) through the lens of language philosophy and psychoanalysis. By incorporating Lacan's concepts of the "chain of signifiers" and "suture points," we propose the Anchor-RAG framework as a novel approach to mitigate hallucinations. In contrast to the predominant reliance on trial-and-error experiments, constant adjustments of mathematical formulas, or resource-intensive methods that emphasize quantity over quality, our approach returns to the fundamental principles of linguistics to analyze the root causes of hallucinations in LLMs. Drawing from robust theoretical foundations, we derive algorithms and models that are not only effective in reducing hallucinations but also enhance LLM performance and improve output quality. This paper seeks to establish a comprehensive theoretical framework for understanding hallucinations in LLMs and aims to challenge the prevalent "guess-and-test" approach and rat race mentality in the field. We aspire to pave the way for a new era of interpretable LLMs, offering deeper insights into the inner workings of language-based AI systems.