Trapping LLM Hallucinations Using Tagged Context Prompts
Philip Feldman, James R. Foulds, Shimei Pan
TL;DR
The paper tackles LLM hallucinations by introducing tagged-context prompts that anchor responses to provided sources. Through a multi-stage methodology—data creation, verification, tag placement, and cross-model experiments—the authors demonstrate that including contextual information dramatically reduces hallucinations and that embedding source tags within contexts enables automated grounding with high reliability. The approach achieves a near-complete elimination of URL-based hallucinations in tagged contexts and shows substantial improvements even with mismatched contexts, though it acknowledges vulnerabilities to poisoned inputs and adversarial prompts. These findings offer a practical, explainable mechanism to improve the trustworthiness of LLM outputs in real-world applications.
Abstract
Recent advances in large language models (LLMs), such as ChatGPT, have led to highly sophisticated conversation agents. However, these models suffer from "hallucinations," where the model generates false or fabricated information. Addressing this challenge is crucial, particularly with AI-driven platforms being adopted across various sectors. In this paper, we propose a novel method to recognize and flag instances when LLMs perform outside their domain knowledge, and ensuring users receive accurate information. We find that the use of context combined with embedded tags can successfully combat hallucinations within generative language models. To do this, we baseline hallucination frequency in no-context prompt-response pairs using generated URLs as easily-tested indicators of fabricated data. We observed a significant reduction in overall hallucination when context was supplied along with question prompts for tested generative engines. Lastly, we evaluated how placing tags within contexts impacted model responses and were able to eliminate hallucinations in responses with 98.88% effectiveness.