Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Does Object Grounding Really Reduce Hallucination of Large Vision-Language Models?

Gregor Geigle, Radu Timofte, Goran Glavaš

TL;DR

This paper interrogates the commonly held claim that object grounding reduces hallucination in large vision-language models. By evaluating two prevalent grounding objectives—referring expressions and grounded captioning—across three backbones with open-ended caption generation and out-of-distribution data, it shows grounding offers little to no reduction in object hallucination. The study employs QA-based (POPE) and open-generation metrics (CHAIR, FaithScore), along with a semantic extension CHAIR-MEN, to provide a robust assessment beyond MSCOCO-bound evaluations. Inference-time grounding yields only modest improvements at the cost of caption informativeness, calling for new mitigation strategies beyond grounding. The work thus challenges assumptions about grounding as a primary tool for hallucination reduction in LVLMs and highlights the need for broader methodological innovations.

Abstract

Large vision-language models (LVLMs) have recently dramatically pushed the state of the art in image captioning and many image understanding tasks (e.g., visual question answering). LVLMs, however, often \textit{hallucinate} and produce captions that mention concepts that cannot be found in the image. These hallucinations erode the trustworthiness of LVLMs and are arguably among the main obstacles to their ubiquitous adoption. Recent work suggests that addition of grounding objectives -- those that explicitly align image regions or objects to text spans -- reduces the amount of LVLM hallucination. Although intuitive, this claim is not empirically justified as the reduction effects have been established, we argue, with flawed evaluation protocols that (i) rely on data (i.e., MSCOCO) that has been extensively used in LVLM training and (ii) measure hallucination via question answering rather than open-ended caption generation. In this work, in contrast, we offer the first systematic analysis of the effect of fine-grained object grounding on LVLM hallucination under an evaluation protocol that more realistically captures LVLM hallucination in open generation. Our extensive experiments over three backbone LLMs reveal that grounding objectives have little to no effect on object hallucination in open caption generation.

Does Object Grounding Really Reduce Hallucination of Large Vision-Language Models?

TL;DR

This paper interrogates the commonly held claim that object grounding reduces hallucination in large vision-language models. By evaluating two prevalent grounding objectives—referring expressions and grounded captioning—across three backbones with open-ended caption generation and out-of-distribution data, it shows grounding offers little to no reduction in object hallucination. The study employs QA-based (POPE) and open-generation metrics (CHAIR, FaithScore), along with a semantic extension CHAIR-MEN, to provide a robust assessment beyond MSCOCO-bound evaluations. Inference-time grounding yields only modest improvements at the cost of caption informativeness, calling for new mitigation strategies beyond grounding. The work thus challenges assumptions about grounding as a primary tool for hallucination reduction in LVLMs and highlights the need for broader methodological innovations.

Abstract

Large vision-language models (LVLMs) have recently dramatically pushed the state of the art in image captioning and many image understanding tasks (e.g., visual question answering). LVLMs, however, often \textit{hallucinate} and produce captions that mention concepts that cannot be found in the image. These hallucinations erode the trustworthiness of LVLMs and are arguably among the main obstacles to their ubiquitous adoption. Recent work suggests that addition of grounding objectives -- those that explicitly align image regions or objects to text spans -- reduces the amount of LVLM hallucination. Although intuitive, this claim is not empirically justified as the reduction effects have been established, we argue, with flawed evaluation protocols that (i) rely on data (i.e., MSCOCO) that has been extensively used in LVLM training and (ii) measure hallucination via question answering rather than open-ended caption generation. In this work, in contrast, we offer the first systematic analysis of the effect of fine-grained object grounding on LVLM hallucination under an evaluation protocol that more realistically captures LVLM hallucination in open generation. Our extensive experiments over three backbone LLMs reveal that grounding objectives have little to no effect on object hallucination in open caption generation.
Paper Structure (14 sections, 2 figures, 7 tables)

This paper contains 14 sections, 2 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Grounding Objectives in LVLMs
  3. Objectives.
  4. Measuring Object Hallucination
  5. Experimental Setup
  6. Results
  7. Qualitative Grounded Caption Analysis
  8. Related Work
  9. Hallucination Mitigation.
  10. Conclusion
  11. Limitations
  12. Training and Details
  13. Long Captions
  14. CHAIR and CHAIR-MEN

Figures (2)

  • Figure 1: CHAIR and FaithScore are used to measure hallucinations in open caption generation with LVLMs. CHAIR relies on human object annotation (over a fixed set) to identify objects and check if they are hallucinated. FaithScore first uses an LLM to convert captions into facts which are then verified by a VQA model.
  • Figure 2: Qualitative examples of Vicuna +RE+GC for standard and grounded captioning. Hallucinations are underlined in red. Predicted bounding boxes are visualized in the image and marked in the caption.