Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From Detection to Diagnosis: Advancing Hallucination Analysis with Automated Data Synthesis

Yanyi Liu, Qingwen Yang, Tiezheng Guo, Feiyu Qu, Jun Liu, Yingyou Wen

TL;DR

The paper tackles hallucinations in large language models by shifting from mere detection to a holistic diagnosis framework. It introduces the Hallucination Diagnosis Task and builds an automated Hallucination Diagnosis Generator (HDG) to synthesize richly annotated data, then trains HDM-4B-RL via Group Relative Policy Optimization (GRPO) to output structured diagnostics, localization of hallucinations, explanations, and corrected content. The 4B HDM-4B-RL achieves state-of-the-art performance on detection benchmarks and strong performance on full diagnosis tasks, approaching or matching larger models while maintaining efficiency. The work demonstrates the feasibility and value of end-to-end hallucination diagnosis for building more trustworthy generative AI systems, with implications for safer deployment in high-stakes domains.

Abstract

Hallucinations in Large Language Models (LLMs), defined as the generation of content inconsistent with facts or context, represent a core obstacle to their reliable deployment in critical domains. Current research primarily focuses on binary "detection" approaches that, while capable of identifying hallucinations, fail to provide interpretable and actionable feedback for model improvement, thus limiting practical utility. To address this limitation, a new research paradigm is proposed, shifting from "detection" to "diagnosis". The Hallucination Diagnosis Task is introduced, a task which requires models to not only detect hallucinations, but also perform error localization, causal explanation, and content correction. We develop the Hallucination Diagnosis Generator (HDG), an automated pipeline that systematically generates high-quality training samples with rich diagnostic metadata from raw corpora through multi-dimensional augmentation strategies including controlled fact fabrication and reasoning chain perturbation. Using HDG-generated data, we train HDM-4B-RL, a 4-billion-parameter hallucination diagnosis model, employing Group Relative Policy Optimization (GRPO) with a comprehensive reward function incorporating structural, accuracy, and localization signals. Experimental results demonstrate that our model surpasses previous state-of-the-art detection models on the HaluEval benchmark while achieving comparable performance to advanced general-purpose models. In comprehensive diagnosis tasks, HDM-4B-RL matches the capabilities of larger general models while maintaining a smaller size. This work validates the feasibility and value of hallucination diagnosis, providing an effective methodology for building more trustworthy and reliable generative AI systems.

From Detection to Diagnosis: Advancing Hallucination Analysis with Automated Data Synthesis

TL;DR

The paper tackles hallucinations in large language models by shifting from mere detection to a holistic diagnosis framework. It introduces the Hallucination Diagnosis Task and builds an automated Hallucination Diagnosis Generator (HDG) to synthesize richly annotated data, then trains HDM-4B-RL via Group Relative Policy Optimization (GRPO) to output structured diagnostics, localization of hallucinations, explanations, and corrected content. The 4B HDM-4B-RL achieves state-of-the-art performance on detection benchmarks and strong performance on full diagnosis tasks, approaching or matching larger models while maintaining efficiency. The work demonstrates the feasibility and value of end-to-end hallucination diagnosis for building more trustworthy generative AI systems, with implications for safer deployment in high-stakes domains.

Abstract

Hallucinations in Large Language Models (LLMs), defined as the generation of content inconsistent with facts or context, represent a core obstacle to their reliable deployment in critical domains. Current research primarily focuses on binary "detection" approaches that, while capable of identifying hallucinations, fail to provide interpretable and actionable feedback for model improvement, thus limiting practical utility. To address this limitation, a new research paradigm is proposed, shifting from "detection" to "diagnosis". The Hallucination Diagnosis Task is introduced, a task which requires models to not only detect hallucinations, but also perform error localization, causal explanation, and content correction. We develop the Hallucination Diagnosis Generator (HDG), an automated pipeline that systematically generates high-quality training samples with rich diagnostic metadata from raw corpora through multi-dimensional augmentation strategies including controlled fact fabrication and reasoning chain perturbation. Using HDG-generated data, we train HDM-4B-RL, a 4-billion-parameter hallucination diagnosis model, employing Group Relative Policy Optimization (GRPO) with a comprehensive reward function incorporating structural, accuracy, and localization signals. Experimental results demonstrate that our model surpasses previous state-of-the-art detection models on the HaluEval benchmark while achieving comparable performance to advanced general-purpose models. In comprehensive diagnosis tasks, HDM-4B-RL matches the capabilities of larger general models while maintaining a smaller size. This work validates the feasibility and value of hallucination diagnosis, providing an effective methodology for building more trustworthy and reliable generative AI systems.
Paper Structure (36 sections, 4 equations, 4 figures, 10 tables, 1 algorithm)

This paper contains 36 sections, 4 equations, 4 figures, 10 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Hallucination Detection
  4. Reasoning in Large Language Models
  5. Methodology
  6. Hallucination Diagnosis Generator (HDG)
  7. Step 1: Task-Oriented Seed Sample Generation
  8. Step 2: Sample Augmentation
  9. Step 3: Sample Quality Verification
  10. Step 4: Metadata Enrichment
  11. Hallucination Diagnosis Model
  12. Structured Output Reward ($R_{struct}$)
  13. Detection Accuracy Reward ($R_{acc}$)
  14. Localization Reward ($R_{loc}$)
  15. Final Reward Aggregation
  16. ...and 21 more sections

Figures (4)

  • Figure 1: Contrasting hallucination detection with diagnosis.
  • Figure 2: An overview of the Hallucination Diagnosis Generator (HDG) pipeline, detailing its four core stages from raw text to richly annotated diagnostic data.
  • Figure 3: Distribution of reasoning steps in the synthesized hallucination diagnosis dataset, categorized by task type.
  • Figure 4: Distribution of context lengths in the constructed hallucination diagnosis dataset.