Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMs

TL;DR

The paper addresses the mismatch between visual perception and correct reasoning in vision-language models by analyzing layer-wise attention dynamics and identifying a consistent shift from text processing in early layers to visual grounding in deeper layers. It reveals a 'seeing but believing' phenomenon where models attend to the right evidence even when answers are incorrect, motivating an inference-time solution. The authors introduce Vea, a training-free Visual Evidence Augmentation that highlights deep-layer evidence regions and improves grounding and QA accuracy across multiple VLM families and datasets. This work offers a diagnostic framework and a practical, broadly applicable method to bridge perception and reasoning, enhancing reliability and interpretability in multimodal AI.

Abstract

Vision-Language Models (VLMs) achieve strong results on multimodal tasks such as visual question answering, yet they can still fail even when the correct visual evidence is present. In this work, we systematically investigate whether these failures arise from not perceiving the evidence or from not leveraging it effectively. By examining layer-wise attention dynamics, we find that shallow layers focus primarily on text, while deeper layers sparsely but reliably attend to localized evidence regions. Surprisingly, VLMs often perceive the visual evidence when outputting incorrect answers, a phenomenon we term ``seeing but not believing'' that widely exists in major VLM families. Building on this, we introduce an inference-time intervention that highlights deep-layer evidence regions through selective attention-based masking. It requires no training and consistently improves accuracy across multiple families, including LLaVA, Qwen, Gemma, and InternVL. These results show that VLMs encode reliable evidence internally but under-utilize it, making such signals explicit can bridge the gap between perception and reasoning, advancing the diagnostic understanding and reliability of VLMs.

Figures (10)

• Figure 1: Relative attention per token (RAPT)$^1$ (y-axis) to text tokens (red) and image tokens (blue) across model layers (x-axis). While early layers strongly emphasize text, deeper layers progressively increase attention to images, showing a sequential transition from linguistic parsing to visual grounding within a single-token inference. Similar trends hold across different VLM families, please see Appendix \ref{['sec:ap-res']} for more results.
• Figure 2: Visualization of image attention across different layers on VQA samples. Red bounding boxes denote ground-truth evidence regions. While shallow layers exhibit global weak attention, deeper layers consistently focus on localized regions that align with the relevant evidence.
• Figure 3: Qualitative examples of "seeing but not believing". Each case shows the input image (left) and the average attention map of the late 50% layers (right). Deep layers attend to the correct evidence regions (red boxes), but the final answers are still incorrect due to generation failures.
• Figure 4: Relative attention to the evidence/non-evidence image tokens (y-axis) across the layers (x-axis) for different VLM families. Deeper layers pay much greater attention to crucial evidence (blue lines) in the context, even when VLM responds incorrectly (dashed lines). Best viewed in color.
• Figure 5: Overview of the proposed Vea framework. Best viewed in color.