AutoHallusion: Automatic Generation of Hallucination Benchmarks for Vision-Language Models

TL;DR

The development of AutoHallusion, the first automated benchmark generation approach that employs several key strategies to create a diverse range of hallucination examples, enables us to create new benchmarks at the minimum cost and thus overcomes the fragility of hand-crafted benchmarks.

Abstract

Large vision-language models (LVLMs) are prone to hallucinations, where certain contextual cues in an image can trigger the language module to produce overconfident and incorrect reasoning about abnormal or hypothetical objects. While some benchmarks have been developed to investigate LVLM hallucinations, they often rely on hand-crafted corner cases whose failure patterns may not generalize well. Additionally, fine-tuning on these examples could undermine their validity. To address this, we aim to scale up the number of cases through an automated approach, reducing human bias in crafting such corner cases. This motivates the development of AutoHallusion, the first automated benchmark generation approach that employs several key strategies to create a diverse range of hallucination examples. Our generated visual-question pairs pose significant challenges to LVLMs, requiring them to overcome contextual biases and distractions to arrive at correct answers. AutoHallusion enables us to create new benchmarks at the minimum cost and thus overcomes the fragility of hand-crafted benchmarks. It also reveals common failure patterns and reasons, providing key insights to detect, avoid, or control hallucinations. Comprehensive evaluations of top-tier LVLMs, e.g., GPT-4V(ision), Gemini Pro Vision, Claude 3, and LLaVA-1.5, show a 97.7% and 98.7% success rate of hallucination induction on synthetic and real-world datasets of AutoHallusion, paving the way for a long battle against hallucinations. The codebase and data can be accessed at https://github.com/wuxiyang1996/AutoHallusion.

Figures (13)

• Figure 1: AutoHallusion: We propose three image manipulation strategies to induce hallucinations: abnormal object insertion, paired object insertion, and correlated object removal, which trigger the conflicts between the images and LVLM priors. Given generated images, we ask LVLMs questions on object existence and their spatial relations for visual question answering.
• Figure 2: Overview of AutoHallusion. We first automatically generate the scenes set and objects set (pink). After that, we use text to probe the language prior of the victim LVLM and then propose three manipulation strategies to induce hallucination in scene images(yellow). We finally use two metrics to detect hallucinations (blue).
• Figure 3: Hallucination Cases created by AutoHallusion. We highlight hallucination context made by Correctness and Inconsistency
• Figure 4: Ablation on using different LVLMs for object prompting and VQA tasks.
• Figure 5: Hallucination Cases Created by Abnormal Object Insertion: We highlight hallucination context made by Correctness , Inconsistency , or potentially mixed .