Hallucination Begins Where Saliency Drops

TL;DR

The paper tackles hallucinations in large vision-language models by introducing LVLMs-Saliency, a gradient-aware diagnostic that combines attention with input gradients to quantify token grounding. It reveals a strong link between hallucinations and low saliency of preceding tokens, indicating breakdowns in contextual memory. To mitigate this, it proposes Saliency-Guided Rejection Sampling (SGRS) and Local Coherence Reinforcement (LocoRE), which filter low-saliency candidates during decoding and reinforce local textual coherence, respectively. Across multiple LVLMs and benchmarks, SGRS+LocoRE substantially reduces hallucinations while preserving fluency and task performance, without requiring model retraining, and the authors provide code for replication.

Abstract

Recent studies have examined attention dynamics in large vision-language models (LVLMs) to detect hallucinations. However, existing approaches remain limited in reliably distinguishing hallucinated from factually grounded outputs, as they rely solely on forward-pass attention patterns and neglect gradient-based signals that reveal how token influence propagates through the network. To bridge this gap, we introduce LVLMs-Saliency, a gradient-aware diagnostic framework that quantifies the visual grounding strength of each output token by fusing attention weights with their input gradients. Our analysis uncovers a decisive pattern: hallucinations frequently arise when preceding output tokens exhibit low saliency toward the prediction of the next token, signaling a breakdown in contextual memory retention. Leveraging this insight, we propose a dual-mechanism inference-time framework to mitigate hallucinations: (1) Saliency-Guided Rejection Sampling (SGRS), which dynamically filters candidate tokens during autoregressive decoding by rejecting those whose saliency falls below a context-adaptive threshold, thereby preventing coherence-breaking tokens from entering the output sequence; and (2) Local Coherence Reinforcement (LocoRE), a lightweight, plug-and-play module that strengthens attention from the current token to its most recent predecessors, actively counteracting the contextual forgetting behavior identified by LVLMs-Saliency. Extensive experiments across multiple LVLMs demonstrate that our method significantly reduces hallucination rates while preserving fluency and task performance, offering a robust and interpretable solution for enhancing model reliability. Code is available at: https://github.com/zhangbaijin/LVLMs-Saliency

Figures (11)

• Figure 1: Attention vs. Saliency Maps for Correct and Hallucinated Tokens (Qwen2-VL-7B). Left (correct token wallpaper): Attention maps show no distinctive pattern, while our LVLMs-Saliency maps reveal strong, structured grounding to prior outputs. Right (hallucinated token blue): Attention maps remain visually similar, but saliency maps collapse, signaling loss of contextual dependency.
• Figure 2: Output Token Saliency Patterns in Qwen2-VL-7B. When generating a correct token (e.g., wallpaper), the current token assigns high saliency to recent output tokens, typically decaying with distance. In contrast, when generating a hallucinated token (e.g., blue), saliency toward all prior outputs collapses — signaling contextual disconnection.
• Figure 3: Effect of LocoRE on output token saliency map (Qwen2-VL-7B).Without LocoRE: When generating an incorrect token(clock), saliency scores assigned to prior output tokens are low — indicating weak contextual grounding. With LocoRE: The same position now generates a correct token(watch), accompanied by significantly higher saliency scores to recent outputs — demonstrating LocoRE’s ability to restore contextual coherence and prevent hallucination via attention reinforcement.
• Figure 4: Ablation study of $\alpha$: trade-offs between hallucination rate, recall, and latency.
• Figure 5: Attention map and saliency map of LLaVA1.5-7B.