HalluClean: A Unified Framework to Combat Hallucinations in LLMs

TL;DR

Hallucinations in LLM outputs undermine factual reliability across many tasks. HalluClean presents a zero-shot, task-agnostic framework that uses structured reasoning and a plan-and-solve style to detect and revise hallucinations without external knowledge sources. The method demonstrates strong, cross-task performance on QA, dialogue, summarization, math word problems, and self-contradiction detection, with domain robustness and compatibility with open-source models. The work provides interpretable, modular components and releases prompts and resources to support practical deployment in privacy-sensitive and resource-constrained settings.

Abstract

Large language models (LLMs) have achieved impressive performance across a wide range of natural language processing tasks, yet they often produce hallucinated content that undermines factual reliability. To address this challenge, we introduce HalluClean, a lightweight and task-agnostic framework for detecting and correcting hallucinations in LLM-generated text. HalluClean adopts a reasoning-enhanced paradigm, explicitly decomposing the process into planning, execution, and revision stages to identify and refine unsupported claims. It employs minimal task-routing prompts to enable zero-shot generalization across diverse domains, without relying on external knowledge sources or supervised detectors. We conduct extensive evaluations on five representative tasks-question answering, dialogue, summarization, math word problems, and contradiction detection. Experimental results show that HalluClean significantly improves factual consistency and outperforms competitive baselines, demonstrating its potential to enhance the trustworthiness of LLM outputs in real-world applications.

Figures (11)

• Figure 1: Overview of the HalluClean framework. It consists of two modules: hallucination detection and revision. The detection module generates a task-specific plan, performs step-by-step reasoning, and makes a final judgment. If a hallucination is detected, the revision module revises the content based on the identified reasoning to eliminate hallucinated information.
• Figure 2: Detection adaptability across backbone LLMs. F1 and Acc denote the F1 score and accuracy of hallucination detection, respectively.
• Figure 3: Revision module adaptability across backbone LLMs. R represents hallucination reduction rate, and Q reprensents revision success rate.
• Figure 4: An illustrative example of the question answering hallucination detection and revision. original hallucinated content, correct conclusions, and correct reasoning and revisions.
• Figure 5: An illustrative example of the dialogue system hallucination detection and revision.original hallucinated content, correct conclusions, and correct reasoning and revisions.