Visual Hallucination: Definition, Quantification, and Prescriptive Remediations

TL;DR

This work addresses visual hallucination in vision-language models by proposing eight fine-grained categories and introducing VHILT, a 2,000-sample benchmark across image captioning and VQA with multi-model annotations. It documents a meticulous dataset construction, including model selection (Kosmos-2, MiniGPT-V2, Sphinx for captioning; LLaVa, MiniGPT-4, InstructBLIP, MultimodalGPT, MplugOwl for VQA) and a rigorous annotation pipeline with in-house and AMT components. The paper also surveys a taxonomy of mitigation strategies—data-driven, training adjustments, and post-processing—and discusses representative methods to guide future remediation efforts. The VHILT resource, paired with a clarified taxonomy, provides a practical platform for researchers and developers to quantify, analyze, and reduce visual hallucinations in VLMs, accelerating safer deployment of multimodal AI systems.

Abstract

The troubling rise of hallucination presents perhaps the most significant impediment to the advancement of responsible AI. In recent times, considerable research has focused on detecting and mitigating hallucination in Large Language Models (LLMs). However, it's worth noting that hallucination is also quite prevalent in Vision-Language models (VLMs). In this paper, we offer a fine-grained discourse on profiling VLM hallucination based on two tasks: i) image captioning, and ii) Visual Question Answering (VQA). We delineate eight fine-grained orientations of visual hallucination: i) Contextual Guessing, ii) Identity Incongruity, iii) Geographical Erratum, iv) Visual Illusion, v) Gender Anomaly, vi) VLM as Classifier, vii) Wrong Reading, and viii) Numeric Discrepancy. We curate Visual HallucInation eLiciTation (VHILT), a publicly available dataset comprising 2,000 samples generated using eight VLMs across two tasks of captioning and VQA along with human annotations for the categories as mentioned earlier.

Figures (14)

• Figure 1: This describes the case of hallucination in the VQA task. The picture is processed by MiniGPT-4, and the model is asked about the content of the image. The response provided is "GRILL," falling into the hallucination category of Wrong Reading.
• Figure 2: In the VQA task, we display examples across three categories, showcasing instances where the model produces hallucinatory outputs. Explanations for these are provided in each figure, with additional examples detailed in Appendix section \ref{['App:exampleset from VQA']}.
• Figure 3: Taxonomy of hallucination mitigation techniques in VLMs, showcasing various data-driven, training adjustments and post-processing techniques for mitigating hallucinations in VLMs.
• Figure 4: Web interface used to annotate the VHILT dataset using Amazon Mechanical Turk.
• Figure 5: Web interface used to annotate the VHILT dataset using Amazon Mechanical Turk.