Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs

TL;DR

This paper addresses hallucinations in large vision-language models by identifying a phenomenon called text inertia, where generated text remains biased by language priors even when visual input is available. It proposes a training-free method, Pay Attention to Image (PAI), that enhances image-token influence via self-attention modulation and counters language priors through an image-centric logit refinement expressed as $p_{\rm{model}} = \gamma p_{\rm{model}}(\mathbf{y}|\mathbf{X}_V,\mathbf{X}_I,\mathbf{X}_H) - (\gamma-1) p_{\rm{model}}(\mathbf{y}|\mathbf{X}_I,\mathbf{X}_H)$. The approach operates during inference by amplifying image attention in selected decoder layers (guided by a layer-prior) and subtracting text-only logits, all without additional training. Extensive experiments on multiple LVLMs (LLAVA, Shikra, MiniGPT4) across CHAIR, POPE, MMHal-Bench, and GPT-4V evaluations show that PAI substantially reduces hallucinations while preserving efficiency, with ablations highlighting the roles of hyperparameters $\alpha$, $\gamma$, and the layer-prior. Limitations include dependence on LLaMA-family decoders and potential gains from integrating these ideas into training; nonetheless, PAI offers a practical, training-free path to more faithful multimodal generation.

Abstract

Existing Large Vision-Language Models (LVLMs) primarily align image features of vision encoder with Large Language Models (LLMs) to leverage their superior text generation capabilities. However, the scale disparity between vision encoder and language model may led to LLMs assuming a predominant role in multi-modal comprehension. This imbalance in LVLMs may result in the instances of hallucinatory. Concretely, LVLMs may generate consistent descriptions with or without visual input, indicating that certain outputs are influenced solely by context text. We refer to this phenomenon as "text inertia." To counteract this issue, we introduce a training-free algorithm to find an equilibrium point between image comprehension and language inference. Specifically, we adaptively involve adjusting and amplifying the attention weights assigned to image tokens, thereby granting greater prominence to visual elements. Meanwhile, we subtract the logits of multi-modal inputs from ones of pure text input, which can help LVLMs be not biased towards LLMs. By enhancing images tokens and reducing the stubborn output of LLM, we can let LVLM pay more attention to images, towards alleviating text inertia and reducing the hallucination in LVLMs. Our extensive experiments shows that this method substantially reduces the frequency of hallucinatory outputs in various LVLMs in terms of different metrics. Project page is available at https://lalbj.github.io/projects/PAI/.

Figures (10)

• Figure 1: We present an examination of various input settings, with hallucinations specifically highlighted in red. (a) When using LLAVA for image description, it generates a hallucinated description. (b) Even without image input, when only the historical response preceding the hallucinated description is input to LLAVA, it reproduces the same hallucinated description, a phenomenon we refer to as "Text Inertia". (c) Our proposed method, PAI, effectively mitigates this text inertia problem and yields accurate descriptions. (d) Utilizing PAI for image description results in a significantly more precise description.
• Figure 2: Percentage of text inertia hallucination in all hallucination (calculated with 500 samples). For specific calculation processes, please refer to the supplementary material section A.
• Figure 3: Visualization of the average attention ratio of different content. The x-axis denotes the sequence length of the history tokens. The lengths of the image, instruction, BOS, and outlier tokens outlier are all fixed as they are part of the model input, with 576 tokens for the image, 21 for the instruction, and 1 each for the BOS and outlier tokens.
• Figure 4: Architecture of our PAI. To alleviate text inertia, we additionally construct an input without image. Throughout the forward inference process, we amplify the focus on the image token by edit self-attention maps in LLaMA. Ultimately, we subtract the logits distribution of the language prior during decoding to achieve an accurate description.
• Figure 5: The BOS token attention ratios of three model. We calculate the attention weights of the BOS token for each head in each layer and display them in a heat map.