Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ZINA: Multimodal Fine-grained Hallucination Detection and Editing

Yuiga Wada, Kazuki Matsuda, Komei Sugiura, Graham Neubig

Abstract

Multimodal Large Language Models (MLLMs) often generate hallucinations, where the output deviates from the visual content. Given that these hallucinations can take diverse forms, detecting hallucinations at a fine-grained level is essential for comprehensive evaluation and analysis. To this end, we propose a novel task of multimodal fine-grained hallucination detection and editing for MLLMs. Moreover, we propose ZINA, a novel method that identifies hallucinated spans at a fine-grained level, classifies their error types into six categories, and suggests appropriate refinements. To train and evaluate models for this task, we construct VisionHall, a dataset comprising 6.9k outputs from twelve MLLMs manually annotated by 211 annotators, and 20k synthetic samples generated using a graph-based method that captures dependencies among error types. We demonstrated that ZINA outperformed existing methods, including GPT-4o and Llama-3.2, in both detection and editing tasks.

ZINA: Multimodal Fine-grained Hallucination Detection and Editing

Abstract

Multimodal Large Language Models (MLLMs) often generate hallucinations, where the output deviates from the visual content. Given that these hallucinations can take diverse forms, detecting hallucinations at a fine-grained level is essential for comprehensive evaluation and analysis. To this end, we propose a novel task of multimodal fine-grained hallucination detection and editing for MLLMs. Moreover, we propose ZINA, a novel method that identifies hallucinated spans at a fine-grained level, classifies their error types into six categories, and suggests appropriate refinements. To train and evaluate models for this task, we construct VisionHall, a dataset comprising 6.9k outputs from twelve MLLMs manually annotated by 211 annotators, and 20k synthetic samples generated using a graph-based method that captures dependencies among error types. We demonstrated that ZINA outperformed existing methods, including GPT-4o and Llama-3.2, in both detection and editing tasks.

Paper Structure

This paper contains 61 sections, 5 equations, 4 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Hallucination detection and editing in LLMs.
  4. Hallucination analysis in MLLMs.
  5. Multimodal Fine-grained Hallucination Detection and Editing
  6. Taxonomy
  7. Task Definition
  8. Evaluation Metrics
  9. Detection task.
  10. Editing task.
  11. Overall performance.
  12. Methodology
  13. Proposed Method: Zina
  14. Synthetic Training Data Curation
  15. Seed descriptions.
  16. ...and 46 more sections

Figures (4)

  • Figure 1: Overview of the proposed task. In contrast to conventional tasks, the model is expected to detect hallucinated spans at a fine-grained level, classify their types based on a taxonomy, and suggest appropriate refinements.
  • Figure 2: Overview of the graph-based synthetic data generation process. We first obtain seed descriptions by leveraging various MLLMs. Subsequently, the Error Insertion module injects errors while considering inter-span dependencies of errors. The Graph-based Augmentation module then constructs a DAG and prunes it to generate diverse training samples.
  • Figure 3: Qualitative results on the VisionHall dataset. Each subfigure shows the image and the edited descriptions generated by GPT-4o and Zina. Edited spans are enclosed in tags; strikethrough text indicates the original hallucinated phrase, while bold text shows the suggested refinement. Red spans indicate errors such as incorrect tagging or missed detections.
  • Figure 4: Annotation interface used for the VisionHall dataset.