Towards Mitigating Hallucinations in Large Vision-Language Models by Refining Textual Embeddings

TL;DR

This work identifies a modality imbalance in large vision-language models, where simply appending visual embeddings to text biases the model toward language and increases hallucinations. It introduces VisAlign, a training-efficient method that refines textual embeddings by integrating average-pooled visual context, promoting balanced cross-modal attention without major architectural changes. Across multiple benchmarks (e.g., MMVP-MLLM, POPE, MERLIN, Mementos, HallusionBench), VisAlign significantly improves visual grounding and reduces hallucinations, and it demonstrates complementary gains when combined with inference-time methods like Visual Contrastive Decoding. The results suggest a practical, generalizable path to more reliable multimodal reasoning in LVLMs, with broad applicability to existing baselines and datasets.

Abstract

In this work, we identify an inherent bias in prevailing LVLM architectures toward the language modality, largely resulting from the common practice of simply appending visual embeddings to the input text sequence. To address this, we propose a simple yet effective method that refines textual embeddings by integrating average-pooled visual features. Our approach demonstrably improves visual grounding and significantly reduces hallucinations on established benchmarks. While average pooling offers a straightforward, robust, and efficient means of incorporating visual information, we believe that more sophisticated fusion methods could further enhance visual grounding and cross-modal alignment. Given that the primary focus of this work is to highlight the modality imbalance and its impact on hallucinations -- and to show that refining textual embeddings with visual information mitigates this issue -- we leave exploration of advanced fusion strategies for future work.

Figures (6)

• Figure 1: Hallucinations in Video-LLaVA lin2023video.
• Figure 2: Top: Architecture of typical LVLMs like Video-LLaVA, which fuse language and vision embeddings by simple concatenation. Bottom: Our modified architecture with a concatenation block that appends the averaged vision embedding to each token embedding, followed by a projection layer. This encourages the model to learn visually informed textual embeddings and better attend to visual input during training.
• Figure 3: Attention score distributions across the first six attention layers of the baseline Video-LLaVA model (top row) and the VisAlign-enhanced model (bottom row). Video-LLaVA concatenates tokens in a fixed order: 35 initial text tokens, followed by 256 visual embeddings, and then the remaining text tokens. In each map, the x-axis denotes attended tokens (keys), and the y-axis denotes attending tokens (queries). Color intensity reflects attention weight: blue indicates low attention, red/white indicates high attention, and dark (near-black) regions indicate masked or negligible attention due to causal masking in autoregressive LVLMs.
• Figure 4: Qualitative results from the MMVP-MLLM Benchmark: Below each image, the baseline model's response is shown first, followed by the response from the model trained with VisAlign.
• Figure 5: Qualitative examples from POPE A-OKVQA, HallusionBench, MMVP, and Mementos benchmarks illustrating various hallucination types. Input prompts are shown in orange, baseline Video-LLaVA outputs in yellow, and VisAlign-enhanced outputs in green. VisAlign consistently improves performance across object, action, attribute, and relation hallucinations.