Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HalluDetect: Detecting, Mitigating, and Benchmarking Hallucinations in Conversational Systems in the Legal Domain

Spandan Anaokar, Shrey Ganatra, Harshvivek Kashid, Swapnil Bhattacharyya, Shruti Nair, Reshma Sekhar, Siddharth Manohar, Rahul Hemrajani, Pushpak Bhattacharyya

TL;DR

This study addresses hallucinations in legal-domain conversational AI by introducing HalluDetect, a multi-turn, LLM-driven detection framework tailored for RAG-based chats. It benchmarks five mitigation architectures (Vanilla, Prompt-engineered, EditorBot, FactChecker, AgentBot) and demonstrates that AgentBot achieves the lowest hallucination rate with the highest token accuracy, while HalluDetect attains a high F1 score and outperforms baselines in precision. The DetectorEval dataset provides a targeted benchmark for factual grounding in consumer-law dialogues, and extensive experiments—including human evaluation and statistical significance testing—validate HalluDetect’s effectiveness and robustness across architectures. The work offers a scalable framework for detecting, mitigating, and benchmarking hallucinations in sensitive domains, with practical implications for reliability and regulatory compliance.

Abstract

Large Language Models (LLMs) are widely used in industry but remain prone to hallucinations, limiting their reliability in critical applications. This work addresses hallucination reduction in consumer grievance chatbots built using LLaMA 3.1 8B Instruct, a compact model frequently used in industry. We develop HalluDetect, an LLM-based hallucination detection system that achieves an F1 score of 68.92% outperforming baseline detectors by 22.47%. Benchmarking five hallucination mitigation architectures, we find that out of them, AgentBot minimizes hallucinations to 0.4159 per turn while maintaining the highest token accuracy (96.13%), making it the most effective mitigation strategy. Our findings provide a scalable framework for hallucination mitigation, demonstrating that optimized inference strategies can significantly improve factual accuracy.

HalluDetect: Detecting, Mitigating, and Benchmarking Hallucinations in Conversational Systems in the Legal Domain

TL;DR

This study addresses hallucinations in legal-domain conversational AI by introducing HalluDetect, a multi-turn, LLM-driven detection framework tailored for RAG-based chats. It benchmarks five mitigation architectures (Vanilla, Prompt-engineered, EditorBot, FactChecker, AgentBot) and demonstrates that AgentBot achieves the lowest hallucination rate with the highest token accuracy, while HalluDetect attains a high F1 score and outperforms baselines in precision. The DetectorEval dataset provides a targeted benchmark for factual grounding in consumer-law dialogues, and extensive experiments—including human evaluation and statistical significance testing—validate HalluDetect’s effectiveness and robustness across architectures. The work offers a scalable framework for detecting, mitigating, and benchmarking hallucinations in sensitive domains, with practical implications for reliability and regulatory compliance.

Abstract

Large Language Models (LLMs) are widely used in industry but remain prone to hallucinations, limiting their reliability in critical applications. This work addresses hallucination reduction in consumer grievance chatbots built using LLaMA 3.1 8B Instruct, a compact model frequently used in industry. We develop HalluDetect, an LLM-based hallucination detection system that achieves an F1 score of 68.92% outperforming baseline detectors by 22.47%. Benchmarking five hallucination mitigation architectures, we find that out of them, AgentBot minimizes hallucinations to 0.4159 per turn while maintaining the highest token accuracy (96.13%), making it the most effective mitigation strategy. Our findings provide a scalable framework for hallucination mitigation, demonstrating that optimized inference strategies can significantly improve factual accuracy.

Paper Structure

This paper contains 62 sections, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. RAG based Chatbots
  5. HalluDetect: LLM-Based Evaluation
  6. DetectorEval Dataset
  7. Experiments
  8. Structure of Chatbot
  9. Human Evaluation of HalluDetect
  10. Comparative Analysis of Chatbot Architectures
  11. Results and Analysis
  12. Quantitative Performance Analysis
  13. Detector Evaluation
  14. Statistical Significance Testing
  15. Evaluation of Chatbots using HalluDetect
  16. ...and 47 more sections

Figures (3)

  • Figure 1: Example of erroneous hallucination detection by LettuceDetect. Red-highlighted polite language and clarifying questions are incorrectly flagged as hallucinations, revealing the model’s difficulty in distinguishing conversational tone from factual inaccuracy.
  • Figure 2: Overview of the architectures for Chatbots, ChatSimulator, and HalluDetect. Each system integrates LLM modules that process specific inputs (green boxes) through prompts (red boxes) to generate outputs (orange boxes)
  • Figure 3: Box Plot of Count of Hallucinations detected by HalluDetect per Chat for each chatbot variant.