Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HaluNet: Multi-Granular Uncertainty Modeling for Efficient Hallucination Detection in LLM Question Answering

Chaodong Tong, Qi Zhang, Jiayang Gao, Lei Jiang, Yanbing Liu, Nannan Sun

TL;DR

HaluNet addresses the problem of hallucinations in QA by leveraging internal uncertainty signals from LLMs. It introduces a lightweight, multi-branch network that fuses token-level log-likelihoods, entropies, and semantic embeddings, trained with scalable LLM-as-a-Judge supervision. The approach achieves strong detection performance across SQuAD, TriviaQA, and NQ-Open in both context-rich and context-free settings, while maintaining low inference cost. Ablation studies show that multi-granular signals, especially embeddings from intermediate layers, are highly informative, supporting the use of layer 20 for robust detection. Overall, HaluNet offers a practical, real-time solution for robust hallucination detection in LLM-based QA systems and sets the stage for future extensions to broader QA tasks.

Abstract

Large Language Models (LLMs) excel at question answering (QA) but often generate hallucinations, including factual errors or fabricated content. Detecting hallucinations from internal uncertainty signals is attractive due to its scalability and independence from external resources. Existing methods often aim to accurately capture a single type of uncertainty while overlooking the complementarity among different sources, particularly between token-level probability uncertainty and the uncertainty conveyed by internal semantic representations, which provide complementary views on model reliability. We present \textbf{HaluNet}, a lightweight and trainable neural framework that integrates multi granular token level uncertainties by combining semantic embeddings with probabilistic confidence and distributional uncertainty. Its multi branch architecture adaptively fuses what the model knows with the uncertainty expressed in its outputs, enabling efficient one pass hallucination detection. Experiments on SQuAD, TriviaQA, and Natural Questions show that HaluNet delivers strong detection performance and favorable computational efficiency, with or without access to context, highlighting its potential for real time hallucination detection in LLM based QA systems.

HaluNet: Multi-Granular Uncertainty Modeling for Efficient Hallucination Detection in LLM Question Answering

TL;DR

HaluNet addresses the problem of hallucinations in QA by leveraging internal uncertainty signals from LLMs. It introduces a lightweight, multi-branch network that fuses token-level log-likelihoods, entropies, and semantic embeddings, trained with scalable LLM-as-a-Judge supervision. The approach achieves strong detection performance across SQuAD, TriviaQA, and NQ-Open in both context-rich and context-free settings, while maintaining low inference cost. Ablation studies show that multi-granular signals, especially embeddings from intermediate layers, are highly informative, supporting the use of layer 20 for robust detection. Overall, HaluNet offers a practical, real-time solution for robust hallucination detection in LLM-based QA systems and sets the stage for future extensions to broader QA tasks.

Abstract

Large Language Models (LLMs) excel at question answering (QA) but often generate hallucinations, including factual errors or fabricated content. Detecting hallucinations from internal uncertainty signals is attractive due to its scalability and independence from external resources. Existing methods often aim to accurately capture a single type of uncertainty while overlooking the complementarity among different sources, particularly between token-level probability uncertainty and the uncertainty conveyed by internal semantic representations, which provide complementary views on model reliability. We present \textbf{HaluNet}, a lightweight and trainable neural framework that integrates multi granular token level uncertainties by combining semantic embeddings with probabilistic confidence and distributional uncertainty. Its multi branch architecture adaptively fuses what the model knows with the uncertainty expressed in its outputs, enabling efficient one pass hallucination detection. Experiments on SQuAD, TriviaQA, and Natural Questions show that HaluNet delivers strong detection performance and favorable computational efficiency, with or without access to context, highlighting its potential for real time hallucination detection in LLM based QA systems.
Paper Structure (18 sections, 8 equations, 5 figures, 2 tables)

This paper contains 18 sections, 8 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Problem Definition
  5. Training Data Construction
  6. Feature Extraction
  7. HaluNet: Unified Hallucination Detection Network
  8. Experiments
  9. Setup
  10. Datasets.
  11. Implementation Details and Baselines.
  12. Evaluation Metrics.
  13. In-Distribution Evaluation
  14. Out-of-Distribution Evaluation
  15. Ablation Studies
  16. ...and 3 more sections

Figures (5)

  • Figure 1: Integrating token-level uncertainty signals simplifies and enhances the learning process for hallucination detection.
  • Figure 2: HaluNet workflow: (a) training data construction, (b) multi-branch feature extraction and fusion, (c) inference and generalization analysis for robust hallucination detection.
  • Figure 3: HaluNet AUROC and RA@50 across training and test sets. Solid and dashed lines indicate AUROC and RA@50 SOTA ($P(\text{True})$), respectively. X-axis: training datasets; Y-axis: metric values; shaded bars mark ID cases.
  • Figure 4: Heatmap visualization of AUROC achieved by HaluNet under CR = 0 for different backbone models on SQuAD, TriviaQA, and NQ.
  • Figure 5: Layer-wise contribution analysis of hallucination detection performance. The best-performing layer is highlighted, and $\Delta$ vs Mean denotes the relative improvement of the optimal layer over the average performance.