Make VLM Recognize Visual Hallucination on Cartoon Character Image with Pose Information

TL;DR

This work tackles semantic structural hallucinations in non-photorealistic cartoon images generated by large text-to-image models. It introduces Pose-Aware In-Context Visual Learning (PA-ICVL), which augments Vision-Language Models with pose information and in-context examples to detect hallucinations without additional parameter tuning. A dedicated cartoon hallucination dataset is built to train and evaluate the approach, and experiments show substantial performance gains over baselines, with GPT-4v and Gemini Pro Vision achieving 78% and 80% detection accuracy respectively. The method demonstrates how external, domain-specific conditioning can adapt open VLMs to NPR tasks and provides a publicly available dataset for further research and benchmarking.

Abstract

Leveraging large-scale Text-to-Image (TTI) models have become a common technique for generating exemplar or training dataset in the fields of image synthesis, video editing, 3D reconstruction. However, semantic structural visual hallucinations involving perceptually severe defects remain a concern, especially in the domain of non-photorealistic rendering (NPR) such as cartoons and pixelization-style character. To detect these hallucinations in NPR, We propose a novel semantic structural hallucination detection system using Vision-Language Model (VLM). Our approach is to leverage the emerging capability of large language model, in-context learning which denotes that VLM has seen some examples by user for specific downstream task, here hallucination detection. Based on in-context learning, we introduce pose-aware in-context visual learning (PA-ICVL) which improve the overall performance of VLM by further inputting visual data beyond prompts, RGB images and pose information. By incorporating pose guidance, we enable VLMs to make more accurate decisions. Experimental results demonstrate significant improvements in identifying visual hallucinations compared to baseline methods relying solely on RGB images. Within selected two VLMs, GPT-4v, Gemini pro vision, our proposed PA-ICVL improves the hallucination detection with 50% to 78%, 57% to 80%, respectively. This research advances a capability of TTI models toward real-world applications by mitigating visual hallucinations via in-context visual learning, expanding their potential in non-photorealistic domains. In addition, it showcase how users can boost the downstream-specialized capability of open VLM by harnessing additional conditions. We collect synthetic cartoon-hallucination dataset with TTI models, this dataset and final tuned VLM will be publicly available.

Figures (15)

• Figure 1: Examples of semantic structural hallucination in cartoon/pixel rendering images generated by TTI models. These hallucination hinder the TTI model to be extended toward applications, additionally requiring that users embark on burdensome process of eliminating hallucination sample in manual. More samples can be found in Appendices \ref{['suppl:cartoon_domain_hallucination']}.
• Figure 2: Structural gap between real hallucination samples from normal prompt and fake hallucination samples from deliberately designed hallucination prompt. We highlight that generated fake hallucination samples (b) (intentionally generated by hallucination prompt) is not enough to imitate real one, limiting to generate large data sample.
• Figure 3: Schematic comparison to leverage machine-generated images between using existing TTI process and using our proposed method with in-context learned VLM for hallucination verification. Our goal is to detect hallucinations in images generated from TTI using VLM.
• Figure 4: Example of PA-ICVL step (top) and detection step (bottom).
• Figure 5: Pipeline of cartoon-hallucination dataset collection (Stage 1) and hallucination detection (Stage 2).