DynHD: Hallucination Detection for Diffusion Large Language Models via Denoising Dynamics Deviation Learning

Abstract

Diffusion large language models (D-LLMs) have emerged as a promising alternative to auto-regressive models due to their iterative refinement capabilities. However, hallucinations remain a critical issue that hinders their reliability. To detect hallucination responses from model outputs, token-level uncertainty (e.g., entropy) has been widely used as an effective signal to indicate potential factual errors. Nevertheless, the fixed-length generation paradigm of D-LLMs implies that tokens contribute unevenly to hallucination detection, with only a small subset providing meaningful signals. Moreover, the evolution trend of uncertainty throughout the diffusion process can also provide important signals, highlighting the necessity of modeling its denoising dynamics for hallucination detection. In this paper, we propose DynHD that bridge these gaps from both spatial (token sequence) and temporal (denoising dynamics) perspectives. To address the information density imbalance across tokens, we propose a semantic-aware evidence construction module that extracts hallucination-indicative signals by filtering out non-informative tokens and emphasizing semantically meaningful ones. To model denoising dynamics for hallucination detection, we introduce a reference evidence generator that learns the expected evolution trajectory of uncertainty evidence, along with a deviation-based hallucination detector that makes predictions by measuring the discrepancy between the observed and reference trajectories. Extensive experiments demonstrate that DynHD consistently outperforms state-of-the-art baselines while achieving higher efficiency across multiple benchmarks and backbone models.

Figures (8)

• Figure 1: Visualization of spatial uncertainty distribution during decoding. Tokens exhibiting the highest entropy spikes serve as the primary indicators of factual instability. On the contrary, intermediate and structural tokens provide limited cues for hallucination detection.
• Figure 2: Entropy dynamics under spatial filtering. (a) Mean trajectories fail to distinguish between samples due to information density imbalance. (b) & (c) Filtered trajectories (top-$K$) reveal distinct convergence: factual samples decay monotonically, while hallucinated ones exhibit late-stage stagnation or rebound. Differences in decay rates between HotpotQA (b) and TriviaQA (c) reflect varying task-specific denoising dynamics.
• Figure 3: Framework overview of DynHD, which constructs a semantic-aware evidence trajectory from uncertainty, and detects hallucination via learning the deviation between the reference and observed trajectories.
• Figure 4: Pareto-frontier of performance vs. efficiency.
• Figure 5: (a) Visualization of attention weights $\omega_t$ on different steps. (b) Top-$k$ entropy trajectories of representative samples on HotpotQA ($T=128$). (c) Performance comparison of different D-LLMS remasking strategies.