Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mitigating Entity-Level Hallucination in Large Language Models

Weihang Su, Yichen Tang, Qingyao Ai, Changyue Wang, Zhijing Wu, Yiqun Liu

TL;DR

This work tackles entity-level hallucination in LLMs by introducing DRAD, a Dynamic Retrieval Augmentation framework that synchronizes retrieval with real-time hallucination detection (RHD) and revises outputs via self-correction using external knowledge (SEK). RHD relies on uncertainty measures, particularly entity-level probability and entropy, to trigger retrieval only when hallucinations are likely, using thresholds $\theta_1$ and $\theta_2$ to control retrieval frequency. SEK formulates targeted search queries around hallucinated spans, retrieves top-$k$ documents from a Wikipedia-based corpus, and regenerates content to correct errors. Empirical results show state-of-the-art hallucination detection and improved downstream QA performance with significantly fewer retrieval calls, demonstrating DRAD’s effectiveness and efficiency for complex information-seeking tasks.

Abstract

The emergence of Large Language Models (LLMs) has revolutionized how users access information, shifting from traditional search engines to direct question-and-answer interactions with LLMs. However, the widespread adoption of LLMs has revealed a significant challenge known as hallucination, wherein LLMs generate coherent yet factually inaccurate responses. This hallucination phenomenon has led to users' distrust in information retrieval systems based on LLMs. To tackle this challenge, this paper proposes Dynamic Retrieval Augmentation based on hallucination Detection (DRAD) as a novel method to detect and mitigate hallucinations in LLMs. DRAD improves upon traditional retrieval augmentation by dynamically adapting the retrieval process based on real-time hallucination detection. It features two main components: Real-time Hallucination Detection (RHD) for identifying potential hallucinations without external models, and Self-correction based on External Knowledge (SEK) for correcting these errors using external knowledge. Experiment results show that DRAD demonstrates superior performance in both detecting and mitigating hallucinations in LLMs. All of our code and data are open-sourced at https://github.com/oneal2000/EntityHallucination.

Mitigating Entity-Level Hallucination in Large Language Models

TL;DR

This work tackles entity-level hallucination in LLMs by introducing DRAD, a Dynamic Retrieval Augmentation framework that synchronizes retrieval with real-time hallucination detection (RHD) and revises outputs via self-correction using external knowledge (SEK). RHD relies on uncertainty measures, particularly entity-level probability and entropy, to trigger retrieval only when hallucinations are likely, using thresholds and to control retrieval frequency. SEK formulates targeted search queries around hallucinated spans, retrieves top- documents from a Wikipedia-based corpus, and regenerates content to correct errors. Empirical results show state-of-the-art hallucination detection and improved downstream QA performance with significantly fewer retrieval calls, demonstrating DRAD’s effectiveness and efficiency for complex information-seeking tasks.

Abstract

The emergence of Large Language Models (LLMs) has revolutionized how users access information, shifting from traditional search engines to direct question-and-answer interactions with LLMs. However, the widespread adoption of LLMs has revealed a significant challenge known as hallucination, wherein LLMs generate coherent yet factually inaccurate responses. This hallucination phenomenon has led to users' distrust in information retrieval systems based on LLMs. To tackle this challenge, this paper proposes Dynamic Retrieval Augmentation based on hallucination Detection (DRAD) as a novel method to detect and mitigate hallucinations in LLMs. DRAD improves upon traditional retrieval augmentation by dynamically adapting the retrieval process based on real-time hallucination detection. It features two main components: Real-time Hallucination Detection (RHD) for identifying potential hallucinations without external models, and Self-correction based on External Knowledge (SEK) for correcting these errors using external knowledge. Experiment results show that DRAD demonstrates superior performance in both detecting and mitigating hallucinations in LLMs. All of our code and data are open-sourced at https://github.com/oneal2000/EntityHallucination.
Paper Structure (33 sections, 9 equations, 3 figures, 6 tables)

This paper contains 33 sections, 9 equations, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Hallucination Detection
  4. Retrieval-augmented Generation
  5. Methodology
  6. Real-time Hallucination Detection
  7. Empirical Investigation
  8. Entity Probability
  9. Entity Entropy
  10. Threshold for Hallucination Detection
  11. Self-correction based on External Knowledge
  12. Formulating Search Queries
  13. External Knowledge Retrieval
  14. Self-Correction
  15. Discussion
  16. ...and 18 more sections

Figures (3)

  • Figure 1: An illustration of our proposed DRAD Framework, which comprises two main components: RHD and SEK Module, highlighted in the diagram with blue and green frames, respectively.
  • Figure 2: An illustration of our proposed Real-time Hallucination Detection (RHD) framework.
  • Figure 3: Visual comparison between DRAD and baselines across all datasets. For the evaluation metric, we choose Accuracy for StrategyQA, EM for 2WikiMultihopQA, and F1 for NQ.