Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Interpreting and Mitigating Hallucination in MLLMs through Multi-agent Debate

Zheng Lin, Zhenxing Niu, Zhibin Wang, Yinghui Xu

TL;DR

This work tackles hallucination in multimodal LLMs by attributing it to deficits in slow-thinking and divergent-thinking. It introduces a self-reflection scheme to promote careful, iterative reasoning and a multi-agent debate framework to foster diverse perspectives, with a judge resolving disagreements. The authors also reinterpret hallucinations to identify the responsible image regions and provide explanations, proposing POPE-R to fix annotation issues and POPE-C to separate creativity from hallucination. Through extensive experiments on POPE and revised POPE-R benchmarks using debaters like Gemini-Pro-Vision and GPT-4o, the approach achieves generalized hallucination mitigation and improved interpretability, while also elevating measured creativity. Overall, the method offers a training-free, black-box solution that enhances reliability and provides insight into the causes of cross-modal errors, with practical implications for evaluating and improving MLLMs in vision-language tasks.

Abstract

MLLMs often generate outputs that are inconsistent with the visual content, a challenge known as hallucination. Previous methods focus on determining whether a generated output is hallucinated, without identifying which image region leads to the hallucination or interpreting why such hallucinations occur. In this paper, we argue that hallucination in MLLMs is partially due to a lack of slow-thinking and divergent-thinking in these models. To address this, we propose adopting a self-reflection scheme to promote slow-thinking. Furthermore, we consider eliminating hallucination as a complex reasoning task and propose a multi-agent debate approach to encourage divergent-thinking. Consequently, our approach can not only mitigate hallucinations but also interpret why they occur and detail the specifics of hallucination. In addition, we propose to distinguish creativity from hallucination in the context of MLLMs, and illustrate how to evaluate MLLMs' creativity capability. Extensive experiments on various benchmarks demonstrate that our approach exhibits generalized hallucinations-mitigating performance across several MLLMs.

Interpreting and Mitigating Hallucination in MLLMs through Multi-agent Debate

TL;DR

This work tackles hallucination in multimodal LLMs by attributing it to deficits in slow-thinking and divergent-thinking. It introduces a self-reflection scheme to promote careful, iterative reasoning and a multi-agent debate framework to foster diverse perspectives, with a judge resolving disagreements. The authors also reinterpret hallucinations to identify the responsible image regions and provide explanations, proposing POPE-R to fix annotation issues and POPE-C to separate creativity from hallucination. Through extensive experiments on POPE and revised POPE-R benchmarks using debaters like Gemini-Pro-Vision and GPT-4o, the approach achieves generalized hallucination mitigation and improved interpretability, while also elevating measured creativity. Overall, the method offers a training-free, black-box solution that enhances reliability and provides insight into the causes of cross-modal errors, with practical implications for evaluating and improving MLLMs in vision-language tasks.

Abstract

MLLMs often generate outputs that are inconsistent with the visual content, a challenge known as hallucination. Previous methods focus on determining whether a generated output is hallucinated, without identifying which image region leads to the hallucination or interpreting why such hallucinations occur. In this paper, we argue that hallucination in MLLMs is partially due to a lack of slow-thinking and divergent-thinking in these models. To address this, we propose adopting a self-reflection scheme to promote slow-thinking. Furthermore, we consider eliminating hallucination as a complex reasoning task and propose a multi-agent debate approach to encourage divergent-thinking. Consequently, our approach can not only mitigate hallucinations but also interpret why they occur and detail the specifics of hallucination. In addition, we propose to distinguish creativity from hallucination in the context of MLLMs, and illustrate how to evaluate MLLMs' creativity capability. Extensive experiments on various benchmarks demonstrate that our approach exhibits generalized hallucinations-mitigating performance across several MLLMs.
Paper Structure (16 sections, 11 figures, 3 tables)

This paper contains 16 sections, 11 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Our Approach
  4. Workflow for Multi-agent Debate
  5. Prompt-engineering for Multi-agent Debate
  6. One Example for Our Debate Workflow
  7. Interpreting Hallucination in MLLM
  8. Revised Hallucination Benchmark
  9. Creativity vs. Hallucination
  10. Evaluation
  11. Implementation
  12. Evaluation Benchmark and Metrics.
  13. Main Results
  14. Creativity
  15. Ablation and Discussions
  16. ...and 1 more sections

Figures (11)

  • Figure 1: Our approach can interpret the causes of an MLLM hallucination: (a) MLLM mistakenly identified a "ski pole" as a "fork" due to limited perceptual ability; (b) MLLM mistakenly identified a "plate" as a "bowl" due to their visual similarity; (c) MLLM mistakenly identified a "handbag" as a "backpack" due to their conceptual similarity; (d) MLLM misunderstands the "orange" in the question as a color; (e) MLLM mistakenly inferred the existence of a "bed" upon perceiving a "mattress"; (f) MLLM mistakenly inferred the existence of a "laptop" upon perceiving a "keyboard". Note that (e) and (f) are due to unreasonable guesses/inferences.
  • Figure 2: Workflow of our multi-agent debate, wherein two debaters and one judge engage in a multi-round discussion addressing a hallucination-related debate question.
  • Figure 3: One example for our debate workflow.
  • Figure 4: There are erroneous annotations and ambiguous samples in the original POPE benchmark.
  • Figure 5: Creativity vs. Hallucination.
  • ...and 6 more figures