Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Black-Box Visual Prompt Engineering for Mitigating Object Hallucination in Large Vision Language Models

Sangmin Woo, Kang Zhou, Yun Zhou, Shuai Wang, Sheng Guan, Haibo Ding, Lin Lee Cheong

TL;DR

This work tackles object hallucination in large vision-language models (LVLMs) by treating LVLMs as black boxes and learning to select from a pool of visual prompts (VPs) to overlay on input images. The BBVPE framework couples a pool of predefined VPs, an object localizer to identify target objects, and a lightweight VP router to predict the most effective VP for a given image, enabling prompt-based mitigation without accessing model internals. The router is trained on image–prompt–response data to approximate per-image Oracle behavior, and inference applies the chosen VP before querying the LVLM. Empirical results on POPE and CHAIR benchmarks across open-source and proprietary LVLMs show BBVPE reducing object hallucination relative to baselines and approaching Oracle performance, with favorable efficiency compared to some prior methods. This work offers a practical, model-agnostic pathway to improve LVLM reliability in real-world deployments, while outlining avenues for extension to synthetic imagery, finer prompts, and question-aware prompting.

Abstract

Large Vision Language Models (LVLMs) often suffer from object hallucination, which undermines their reliability. Surprisingly, we find that simple object-based visual prompting -- overlaying visual cues (e.g., bounding box, circle) on images -- can significantly mitigate such hallucination; however, different visual prompts (VPs) vary in effectiveness. To address this, we propose Black-Box Visual Prompt Engineering (BBVPE), a framework to identify optimal VPs that enhance LVLM responses without needing access to model internals. Our approach employs a pool of candidate VPs and trains a router model to dynamically select the most effective VP for a given input image. This black-box approach is model-agnostic, making it applicable to both open-source and proprietary LVLMs. Evaluations on benchmarks such as POPE and CHAIR demonstrate that BBVPE effectively reduces object hallucination.

Black-Box Visual Prompt Engineering for Mitigating Object Hallucination in Large Vision Language Models

TL;DR

This work tackles object hallucination in large vision-language models (LVLMs) by treating LVLMs as black boxes and learning to select from a pool of visual prompts (VPs) to overlay on input images. The BBVPE framework couples a pool of predefined VPs, an object localizer to identify target objects, and a lightweight VP router to predict the most effective VP for a given image, enabling prompt-based mitigation without accessing model internals. The router is trained on image–prompt–response data to approximate per-image Oracle behavior, and inference applies the chosen VP before querying the LVLM. Empirical results on POPE and CHAIR benchmarks across open-source and proprietary LVLMs show BBVPE reducing object hallucination relative to baselines and approaching Oracle performance, with favorable efficiency compared to some prior methods. This work offers a practical, model-agnostic pathway to improve LVLM reliability in real-world deployments, while outlining avenues for extension to synthetic imagery, finer prompts, and question-aware prompting.

Abstract

Large Vision Language Models (LVLMs) often suffer from object hallucination, which undermines their reliability. Surprisingly, we find that simple object-based visual prompting -- overlaying visual cues (e.g., bounding box, circle) on images -- can significantly mitigate such hallucination; however, different visual prompts (VPs) vary in effectiveness. To address this, we propose Black-Box Visual Prompt Engineering (BBVPE), a framework to identify optimal VPs that enhance LVLM responses without needing access to model internals. Our approach employs a pool of candidate VPs and trains a router model to dynamically select the most effective VP for a given input image. This black-box approach is model-agnostic, making it applicable to both open-source and proprietary LVLMs. Evaluations on benchmarks such as POPE and CHAIR demonstrate that BBVPE effectively reduces object hallucination.
Paper Structure (11 sections, 7 equations, 4 figures, 6 tables)

This paper contains 11 sections, 7 equations, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Black-Box Visual Prompt Engineering
  4. Experiments
  5. Evaluation Results
  6. Key Observations
  7. Analysis
  8. Conclusion
  9. Implementation Details
  10. More Details on Evaluation Setup
  11. Instruction for GPT-4o Evaluation

Figures (4)

  • Figure 1: Motivation.(left) An LVLM misidentifies a zebra as a horse, demonstrating object hallucination. Various VPs elicit different responses, but their effectiveness depends on the specific characteristics of the image. To remove randomness and solely see the impact of visual prompting, all responses are generated using greedy decoding. (right) While most VPs yield comparable performances, an Oracle---which adaptively applies the best-performing VP per image---dramatically boosts results.
  • Figure 2: Overview.(left) BBVPE utilizes a VP router and object localizer to mitigate object hallucinations in LVLMs. VP router dynamically selects the optimal VP for a given image. (right) During its training phase, a set of images with various VPs and a series of object-related questions are posed to the LVLMs. The question set includes both objects that are present and not present in the image. LVLM responses are then evaluated based on accuracy. The VP router predicts scores for each VP, optimizing the selection process to identify the most effective prompt for a given image.
  • Figure 3: Impact of different VPs on image description generation. Different VPs produce varied results, but not all are equally effective. All responses are generated using greedy decoding to eliminate randomness and focus solely on the influence of visual prompting.
  • Figure 4: GPT-4o evaluation instruction.