VOPE: Revisiting Hallucination of Vision-Language Models in Voluntary Imagination Task

TL;DR

VOPE reframes LVLM hallucination evaluation around voluntary imagination rather than solely image-grounded outputs. It uses a recheck-based presence evaluation that separates hallucinations in factual descriptions from errors in voluntary imagination, and it introduces Hal-D, Hal-I, and Exp as core metrics. The study shows most LVLMs have low Hal-D but high Hal-I, and finds that current mitigation methods poorly reduce Hal-I. It also adds a relevance assessment and demonstrates that contrastive decoding can adjust expressive tendency without changing hallucination rates, highlighting directions for improving visual grounding in imagination tasks. $Hal\text{-}D = \frac{|D_H|}{|D_T|+|D_H|}$, $Hal\text{-}I = \frac{|I_H|}{|I_T|+|I_H|}$, and $Exp = \frac{|I_T|+|I_H|}{|D_T|+|D_H|+|I_T|+|I_H|}$ are central to the framework.

Abstract

Most research on hallucinations in Large Vision-Language Models (LVLMs) focuses on factual description tasks that prohibit any output absent from the image. However, little attention has been paid to hallucinations in voluntary imagination tasks, e.g., story writing, where the models are expected to generate novel content beyond the given image. In these tasks, it is inappropriate to simply regard such imagined novel content as hallucinations. To address this limitation, we introduce Voluntary-imagined Object Presence Evaluation (VOPE)-a novel method to assess LVLMs' hallucinations in voluntary imagination tasks via presence evaluation. Specifically, VOPE poses recheck-based questions to evaluate how an LVLM interprets the presence of the imagined objects in its own response. The consistency between the model's interpretation and the object's presence in the image is then used to determine whether the model hallucinates when generating the response. We apply VOPE to several mainstream LVLMs and hallucination mitigation methods, revealing two key findings: (1) most LVLMs hallucinate heavily during voluntary imagination, and their performance in presence evaluation is notably poor on imagined objects; (2) existing hallucination mitigation methods show limited effect in voluntary imagination tasks, making this an important direction for future research.

Figures (13)

• Figure 1: Comparison between VOPE and response-based assessments. In this example, the model outputs an absent object "toy" when inferring about what might have just happened in the image. A response-based assessment such as $\text{CHAIR}_i$ would regard "toy" as a hallucination. In contrast, VOPE method rechecks the model’s interpretation of the output, identifying "toy" as non-hallucinatory voluntary imagination and correcting the calculation of the hallucination rate.
• Figure 2: Overview of the Voluntary-imagined Object Presence Evaluation (VOPE) method. After generating a response based on the prompt, the model is required to evaluate the presence of the objects in the response. All output objects are then categorized into two sets: Factual Description (objects that the model believes to exist and outputs accordingly) and Voluntary Imagination (objects that the model outputs even though it believes they do not exist in the image). We separately assess hallucination within these two sets. Finally, a relevance check identifies models with excessive irrelevant imagination.
• Figure 3: Hallucination rates of mainstream LVLMs. It shows that while many LVLMs have a low hallucination rate in factual descriptions ($Hal\text{-}D$), they still exhibit relatively high hallucinations during voluntary imagination ($Hal\text{-}I$).
• Figure 4: An example of hallucination in voluntary imagination. The model generates an output that includes "books", but in the subsequent presence evaluation, it determines that the "book" does not exist. This reflects the model's insufficient understanding of the image content.
• Figure 5: Comparison of the $\text{CHAIR}_i$ and $Hal\text{-}D$ metrics. Although both metrics perform consistently in the captioning task, the $\text{CHAIR}_i$ metric significantly overestimates the hallucinations of some capable LVLMs (such as GPT-4o) in the writing task.