Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens

TL;DR

Object hallucination in LVLMs is driven by visual information processing concentrated in middle layers, with a two-stage mechanism: visual information enrichment in layers $5\text{-}18$ and semantic refinement in layers $19\text{-}26$. By introducing Visual Attention Ratio (VAR) and a logit-lens analysis, the study shows real object tokens receive stronger image-token attention than hallucinations, enabling SVAR-based detection (AUROC up to $0.74$, mAP $0.88$) and a simple inference-time Heads Guided Attention Intervention that aggregates across heads to mitigate hallucinations without retraining. The method, validated across LLaVA-1.5 (7B/13B), Shikra-7B, and MiniGPT-4-7B, substantially reduces CHAIR$_S$ (up to $24.1$ points) and CHAIR$_I$ (up to $6.3$ points) while maintaining descriptive richness. These findings advocate internal-state calibration—specifically middle-layer attention—as a practical path to more reliable LVLMs, with implications for automatic hallucination detection and broader interpretability of multimodal reasoning.

Abstract

Hallucinations in Large Vision-Language Models (LVLMs) significantly undermine their reliability, motivating researchers to explore the causes of hallucination. However, most studies primarily focus on the language aspect rather than the visual. In this paper, we address how LVLMs process visual information and whether this process causes hallucination. Firstly, we use the attention lens to identify the stages at which LVLMs handle visual data, discovering that the middle layers are crucial. Moreover, we find that these layers can be further divided into two stages: ''visual information enrichment'' and ''semantic refinement'' which respectively propagate visual data to object tokens and interpret it through text. By analyzing attention patterns during the visual information enrichment stage, we find that real tokens consistently receive higher attention weights than hallucinated ones, serving as a strong indicator of hallucination. Further examination of multi-head attention maps reveals that hallucination tokens often result from heads interacting with inconsistent objects. Based on these insights, we propose a simple inference-time method that adjusts visual attention by integrating information across various heads. Extensive experiments demonstrate that this approach effectively mitigates hallucinations in mainstream LVLMs without additional training costs. Code is available at https://github.com/ZhangqiJiang07/middle_layers_indicating_hallucinations.

Figures (22)

• Figure 1: Illustration of our findings: (i) visual information is primarily processed in the middle layers where (a) the model extracts the visual information and then (b) interprets the semantics embedded in image tokens; (ii) for real tokens like "cat", the attention weights over image tokens are generally higher than hallucinated ones like "TV" in (a); (iii) the model may combine the visual features extracted from multiple objects to produce hallucinations.
• Figure 2: (a) Distribution of visual attention ratio for real object tokens across heads and layers in LLaVA-1.5-7B, sorted row-wise by attention ratios. Note that the high attention in the $0$-th layer (bottom row, green rectangle) is not consistent across all LVLMs, as Shikra and MiniGPT-4 models fail to exhibit this pattern, see \ref{['appendix:case_studies']}. (b) The logit contribution of attention sublayers to real token prediction. We find the middle layers continuously assign higher attention weights to image tokens and exhibit two different contribution patterns to the correct prediction.
• Figure 3: Case study of image hidden state interpretation in LLaVA-1.5-7B via logit lens. The heatmap illustrates the retrieved texts of the image hidden states across layers in the three distinct regions, differentiated by color. Our findings reveal that the model's semantic comprehension of image tokens emerges in the later middle layers (19-26) while remaining largely absent in earlier layers. The real and hallucinated object tokens are presented in blue and red in the description, respectively.
• Figure 4: Case study of the visual attention ratio distribution over heads and layers in LLaVA-1.5-7B. Left: real objects; Right: hallucinated objects. We find that the hallucinated object tokens exhibit inactive attention patterns during visual information enrichment compared to the real ones.
• Figure 5: $\textrm{SVAR}_{5\textrm{-}18}$ score distributions across object token types for the 7B (a) and 13B (b) versions of LLaVA-1.5.