Attention Reallocation: Towards Zero-cost and Controllable Hallucination Mitigation of MLLMs

TL;DR

MLLMs frequently hallucinate due to excessive reliance on language priors driven by misallocated self-attention. The authors propose AttnReal, a training-free attention reallocation method that detects attention sinks among output tokens and redistributes their attention to visual tokens, controlled by a down-scaling factor to trade off faithfulness and overall performance. By operating within the forward pass with minimal overhead, AttnReal improves grounding and reduces hallucinations across six open-source MLLMs and three decoding strategies, as demonstrated on CHAIR, MMHal-Bench, and GPT-assisted evaluations. The work highlights the central role of attention distribution in hallucinations and offers a practical, scalable solution for safer, more reliable MLLMs in vision-language tasks.

Abstract

Multi-Modal Large Language Models (MLLMs) stand out in various tasks but still struggle with hallucinations. While recent training-free mitigation methods mostly introduce additional inference overhead via retrospection strategy and contrastive decoding, we propose attention reallocation (AttnReal) to mitigate hallucinations with nearly zero extra cost. Our approach is motivated by the key observations that, MLLM's unreasonable attention distribution causes features to be dominated by historical output tokens, which further contributes to hallucinated responses because of the distribution gap between different token types. Based on the observations, AttnReal recycles excessive attention from output tokens and reallocates it to visual tokens, which reduces MLLM's reliance on language priors and ensures the decoding process depends more on the visual inputs. More interestingly, we find that, by controlling the intensity of AttnReal, we can achieve a wide-range trade-off between the response faithfulness and overall performance. Comprehensive results from different benchmarks validate the effectiveness of AttnReal across six open-source MLLMs and three decoding strategies.

Figures (24)

• Figure 1: Performance comparison for various training-free methods to mitigate MLLM hallucinations on LLaVA-1.5-7B and the CHAIR benchmark using three decoding strategies. Lower CHAIR$_\text{I}$ represents fewer hallucinations. Higher F1 scores mean better overall performance (precision and recall). The curve in each sub-figure is obtained by adjusting the intensity of our method. Our proposed method not only yields superior results than state-of-the-art methods on different decoding strategies, but also achieves an excellent trade-off between hallucination and F1.
• Figure 2: Visualization of our observations obtained with samples from MSCOCO on Qwen2-VL wang2024qwen2. (a) A case of the attention sink phenomenon among output tokens. Red areas represent high attention values. The words corresponding to each token are shown below each column. (b) Changes in MLLM's total attention value to historical output tokens and visual tokens. (c) Visualization of feature distribution of different tokens in the last decoder layer of the MLLM.
• Figure 3: (a) Changes in MLLM's total attention value to historical output tokens and visual tokens when applying our AttnReal. (b) Feature distribution of different tokens in the last decoder layer of the MLLM when applying our AttnReal. (c) The upper part shows the image provided to the MLLM, in which the ground-truth objects are highlighted with different colors. The lower part demonstrates the MLLM's predicted logits for various tokens at different intensities of our AttnReal.
• Figure 4: Comparative results on the MMHal-Bench and LLaVA-1.5-7B. To all metrics, higher scores mean better performance.
• Figure 5: Ablation study on LLaVA-1.5-7B about the effect of different sink thresholds $\mathcal{T}$ on the performance.