Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Drowzee: Metamorphic Testing for Fact-Conflicting Hallucination Detection in Large Language Models

Ningke Li, Yuekang Li, Yi Liu, Ling Shi, Kailong Wang, Haoyu Wang

TL;DR

An extensive and extensible framework that constructs a comprehensive factual knowledge base by crawling sources like Wikipedia, seamlessly integrated into Drowzee, and proposes two semantic-aware oracles that assess the similarity between the semantic structures of the LLM answers and ground truth.

Abstract

Large language models (LLMs) have transformed the landscape of language processing, yet struggle with significant challenges in terms of security, privacy, and the generation of seemingly coherent but factually inaccurate outputs, commonly referred to as hallucinations. Among these challenges, one particularly pressing issue is Fact-Conflicting Hallucination (FCH), where LLMs generate content that directly contradicts established facts. Tackling FCH poses a formidable task due to two primary obstacles: Firstly, automating the construction and updating of benchmark datasets is challenging, as current methods rely on static benchmarks that don't cover the diverse range of FCH scenarios. Secondly, validating LLM outputs' reasoning process is inherently complex, especially with intricate logical relations involved. In addressing these obstacles, we propose an innovative approach leveraging logic programming to enhance metamorphic testing for detecting Fact-Conflicting Hallucinations (FCH). Our method gathers data from sources like Wikipedia, expands it with logical reasoning to create diverse test cases, assesses LLMs through structured prompts, and validates their coherence using semantic-aware assessment mechanisms. Our method generates test cases and detects hallucinations across six different LLMs spanning nine domains, revealing hallucination rates ranging from 24.7% to 59.8%. Key observations indicate that LLMs encounter challenges, particularly with temporal concepts, handling out-of-distribution knowledge, and exhibiting deficiencies in logical reasoning capabilities. The outcomes underscore the efficacy of logic-based test cases generated by our tool in both triggering and identifying hallucinations. These findings underscore the imperative for ongoing collaborative endeavors within the community to detect and address LLM hallucinations.

Drowzee: Metamorphic Testing for Fact-Conflicting Hallucination Detection in Large Language Models

TL;DR

An extensive and extensible framework that constructs a comprehensive factual knowledge base by crawling sources like Wikipedia, seamlessly integrated into Drowzee, and proposes two semantic-aware oracles that assess the similarity between the semantic structures of the LLM answers and ground truth.

Abstract

Large language models (LLMs) have transformed the landscape of language processing, yet struggle with significant challenges in terms of security, privacy, and the generation of seemingly coherent but factually inaccurate outputs, commonly referred to as hallucinations. Among these challenges, one particularly pressing issue is Fact-Conflicting Hallucination (FCH), where LLMs generate content that directly contradicts established facts. Tackling FCH poses a formidable task due to two primary obstacles: Firstly, automating the construction and updating of benchmark datasets is challenging, as current methods rely on static benchmarks that don't cover the diverse range of FCH scenarios. Secondly, validating LLM outputs' reasoning process is inherently complex, especially with intricate logical relations involved. In addressing these obstacles, we propose an innovative approach leveraging logic programming to enhance metamorphic testing for detecting Fact-Conflicting Hallucinations (FCH). Our method gathers data from sources like Wikipedia, expands it with logical reasoning to create diverse test cases, assesses LLMs through structured prompts, and validates their coherence using semantic-aware assessment mechanisms. Our method generates test cases and detects hallucinations across six different LLMs spanning nine domains, revealing hallucination rates ranging from 24.7% to 59.8%. Key observations indicate that LLMs encounter challenges, particularly with temporal concepts, handling out-of-distribution knowledge, and exhibiting deficiencies in logical reasoning capabilities. The outcomes underscore the efficacy of logic-based test cases generated by our tool in both triggering and identifying hallucinations. These findings underscore the imperative for ongoing collaborative endeavors within the community to detect and address LLM hallucinations.
Paper Structure (29 sections, 14 equations, 11 figures, 4 tables, 3 algorithms)

This paper contains 29 sections, 14 equations, 11 figures, 4 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Hallucination Categorization
  4. Logic Programming
  5. Motivating Example
  6. Methodology
  7. Factual Knowledge Extraction
  8. Logical Reasoning
  9. Benchmark Construction
  10. Response Evaluation
  11. Evaluation
  12. Experimental Setup
  13. RQ1: Effectiveness
  14. RQ2: FCH Categorization and Analysis
  15. FCH Categorization.
  16. ...and 14 more sections

Figures (11)

  • Figure 1: A Hallucination Output Example.
  • Figure 2: Examples of Logic Programming.
  • Figure 3: Motivating Example.
  • Figure 4: The Workflow of Drowzee. * The avatar is painted by one of the authors to avoid copyright issues.
  • Figure 5: Entity and Relation Categorization.
  • ...and 6 more figures

Theorems & Definitions (5)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Definition 5