Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Developing a Reliable, Fast, General-Purpose Hallucination Detection and Mitigation Service

Song Wang, Xun Wang, Jie Mei, Yujia Xie, Sean Muarray, Zhang Li, Lingfeng Wu, Si-Qing Chen, Wayne Xiong

TL;DR

The paper addresses hallucination in LLM outputs and presents a production-ready service for detection and mitigation. It uses a three-component pipeline—multi-source detection, iterative rewriting, and multi-source verification—targeting intrinsic hallucinations with low latency and cost. The detection stack combines NER, NLI, and span-based (SBD) signals via a Gradient Boosting ensemble, plus GPT-4-based labeling for data generation and evaluation, while the rewriting module uses GPT-4 with two prompts to balance quality and efficiency. Extensive experiments on offline benchmarks and live production traffic demonstrate effective hallucination detection and mitigation, with production viability demonstrated alongside clear trade-offs in latency and cost.

Abstract

Hallucination, a phenomenon where large language models (LLMs) produce output that is factually incorrect or unrelated to the input, is a major challenge for LLM applications that require accuracy and dependability. In this paper, we introduce a reliable and high-speed production system aimed at detecting and rectifying the hallucination issue within LLMs. Our system encompasses named entity recognition (NER), natural language inference (NLI), span-based detection (SBD), and an intricate decision tree-based process to reliably detect a wide range of hallucinations in LLM responses. Furthermore, we have crafted a rewriting mechanism that maintains an optimal mix of precision, response time, and cost-effectiveness. We detail the core elements of our framework and underscore the paramount challenges tied to response time, availability, and performance metrics, which are crucial for real-world deployment of these technologies. Our extensive evaluation, utilizing offline data and live production traffic, confirms the efficacy of our proposed framework and service.

Developing a Reliable, Fast, General-Purpose Hallucination Detection and Mitigation Service

TL;DR

The paper addresses hallucination in LLM outputs and presents a production-ready service for detection and mitigation. It uses a three-component pipeline—multi-source detection, iterative rewriting, and multi-source verification—targeting intrinsic hallucinations with low latency and cost. The detection stack combines NER, NLI, and span-based (SBD) signals via a Gradient Boosting ensemble, plus GPT-4-based labeling for data generation and evaluation, while the rewriting module uses GPT-4 with two prompts to balance quality and efficiency. Extensive experiments on offline benchmarks and live production traffic demonstrate effective hallucination detection and mitigation, with production viability demonstrated alongside clear trade-offs in latency and cost.

Abstract

Hallucination, a phenomenon where large language models (LLMs) produce output that is factually incorrect or unrelated to the input, is a major challenge for LLM applications that require accuracy and dependability. In this paper, we introduce a reliable and high-speed production system aimed at detecting and rectifying the hallucination issue within LLMs. Our system encompasses named entity recognition (NER), natural language inference (NLI), span-based detection (SBD), and an intricate decision tree-based process to reliably detect a wide range of hallucinations in LLM responses. Furthermore, we have crafted a rewriting mechanism that maintains an optimal mix of precision, response time, and cost-effectiveness. We detail the core elements of our framework and underscore the paramount challenges tied to response time, availability, and performance metrics, which are crucial for real-world deployment of these technologies. Our extensive evaluation, utilizing offline data and live production traffic, confirms the efficacy of our proposed framework and service.
Paper Structure (25 sections, 4 figures, 7 tables)

This paper contains 25 sections, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Hallucination Taxonomy
  4. Hallucination Detection
  5. Hallucination Mitigation
  6. Hallucination Taxonomy and GPT4-based detection
  7. Hallucination Taxonomy
  8. GPT4-based Detection
  9. Hallucination Detection and Mitigation
  10. Hallucination detection methods
  11. NER-based detection
  12. NLI-based detection
  13. Span-based detection
  14. Multi-Source Ensemble
  15. GPT4-based rewriting
  16. ...and 10 more sections

Figures (4)

  • Figure 1: End-2-end hallucination detection and mitigation system
  • Figure 2: Hallucination distribution over entity types
  • Figure 3: Span-based hallucination detection
  • Figure 4: User experience of Hallucination Detection & Mitigation Pipeline in Production