Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Probe-Based Hallucination Detection for Large Language Models

Shize Liang, Hongzhi Wang

TL;DR

This work tackles the challenge of hallucinations in large language models by proposing a token-level hallucination detector built from lightweight nonlinear MLP probes attached to frozen LLM hidden states. It introduces a multi-objective loss to improve token discrimination and span coherence, and develops a layer-position–probe performance model coupled with Bayesian optimization to automatically locate the best probe insertion layer. The approach is evaluated on LongFact, HealthBench, and TriviaQA, where MLP probes outperform linear probes in AUC, Recall, and precision while maintaining real-time performance, and demonstrate strong cross-domain generalization. The proposed framework offers a scalable, efficient, and interpretable path toward improving token-level output validity, with potential to complement retrieval or external verification for robust high-stakes applications.

Abstract

Large language models(LLMs) excel at text generation and knowledge question-answering tasks, but they are prone to generating hallucinated content, severely limiting their application in high-risk domains. Current hallucination detection methods based on uncertainty estimation and external knowledge retrieval suffer from the limitation that they still produce erroneous content at high confidence levels and rely heavily on retrieval efficiency and knowledge coverage. In contrast, probe methods that leverage the model's hidden-layer states offer real-time and lightweight advantages. However, traditional linear probes struggle to capture nonlinear structures in deep semantic spaces.To overcome these limitations, we propose a neural network-based framework for token-level hallucination detection. By freezing language model parameters, we employ lightweight MLP probes to perform nonlinear modeling of high-level hidden states. A multi-objective joint loss function is designed to enhance detection stability and semantic disambiguity. Additionally, we establish a layer position-probe performance response model, using Bayesian optimization to automatically search for optimal probe insertion layers and achieve superior training results.Experimental results on LongFact, HealthBench, and TriviaQA demonstrate that MLP probes significantly outperform state-of-the-art methods in accuracy, recall, and detection capability under low false-positive conditions.

Neural Probe-Based Hallucination Detection for Large Language Models

TL;DR

This work tackles the challenge of hallucinations in large language models by proposing a token-level hallucination detector built from lightweight nonlinear MLP probes attached to frozen LLM hidden states. It introduces a multi-objective loss to improve token discrimination and span coherence, and develops a layer-position–probe performance model coupled with Bayesian optimization to automatically locate the best probe insertion layer. The approach is evaluated on LongFact, HealthBench, and TriviaQA, where MLP probes outperform linear probes in AUC, Recall, and precision while maintaining real-time performance, and demonstrate strong cross-domain generalization. The proposed framework offers a scalable, efficient, and interpretable path toward improving token-level output validity, with potential to complement retrieval or external verification for robust high-stakes applications.

Abstract

Large language models(LLMs) excel at text generation and knowledge question-answering tasks, but they are prone to generating hallucinated content, severely limiting their application in high-risk domains. Current hallucination detection methods based on uncertainty estimation and external knowledge retrieval suffer from the limitation that they still produce erroneous content at high confidence levels and rely heavily on retrieval efficiency and knowledge coverage. In contrast, probe methods that leverage the model's hidden-layer states offer real-time and lightweight advantages. However, traditional linear probes struggle to capture nonlinear structures in deep semantic spaces.To overcome these limitations, we propose a neural network-based framework for token-level hallucination detection. By freezing language model parameters, we employ lightweight MLP probes to perform nonlinear modeling of high-level hidden states. A multi-objective joint loss function is designed to enhance detection stability and semantic disambiguity. Additionally, we establish a layer position-probe performance response model, using Bayesian optimization to automatically search for optimal probe insertion layers and achieve superior training results.Experimental results on LongFact, HealthBench, and TriviaQA demonstrate that MLP probes significantly outperform state-of-the-art methods in accuracy, recall, and detection capability under low false-positive conditions.
Paper Structure (23 sections, 1 theorem, 14 equations, 6 figures, 2 tables, 1 algorithm)

This paper contains 23 sections, 1 theorem, 14 equations, 6 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Hallucination Detection Mechanism
  4. Layer Position Probe Performance Model
  5. Methodology
  6. Probe Training
  7. Problem Definition
  8. MLP Probe
  9. Target Function
  10. Layer-Probe Performance Modeling
  11. Layered Separability Assumption
  12. Probe Performance Function
  13. Probe–Language Modeling Loss Coupling
  14. Joint Optimization Objective
  15. Bayesian Optimization Solution
  16. ...and 8 more sections

Key Result

Theorem 1

Let the continuous bounded function $U(l)$ belong to an RKHS defined by the kernel $k(\cdot,\cdot)$, and assume that observations at layer $l$ are corrupted by sub-Gaussian noise with variance $\sigma^2$, i.e., $\tilde{U}(l) = U(l) + \varepsilon$. Using the GP-UCB strategy where $\mu_{t-1}(l)$ and $\sigma_{t-1}(l)$ are the GP posterior mean and standard deviation, and $\beta_t$ is the confidence

Figures (6)

  • Figure 1: Annotated examples of hallucination detection text, where each token's hallucination detection probe score is highlighted in yellow. The intensity of the color reflects the magnitude of the score: green indicates supported entities, while red denotes entities flagged as hallucinations.
  • Figure 2: A General Framework for Language Model Hallucination Detection Based on Neural Network Probes. The model extracts latent states from frozen intermediate layers of Qwen2.5-7B-Instruct and computes hallucination probabilities for each token via MLP probes to enable real-time detection.
  • Figure 3: Multi-Layer Perceptron Probe Structure
  • Figure 4: Entity-Level Token Annotation Process. Based on the LongFact++ prompt collection across several areas, the LLM produces long-form completions with both factual and imaginative content. After that, it uses an LLM with online search capabilities to find and verify entities in the created content, producing datasets from four different domains.
  • Figure 5: MLP probes versus linear probes across different tasks. Figures (a)–(c) correspond to the dataset performance, language modeling loss, and aggregated label prediction loss, respectively.
  • ...and 1 more figures

Theorems & Definitions (1)

  • Theorem 1: Asymptotic Optimality of Bayesian Optimization