HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data

TL;DR

This paper tackles hallucination toxicity in machine-generated visual instruction data for MLLMs by introducing HalluciDoctor, a data-centric framework that detects hallucinated content via a cross-checking paradigm and eliminates it to produce a rectified dataset (LLaVA+). It further addresses spurious long-tail object co-occurrences through seesaw-based counterfactual instruction expansion, yielding LLaVA++ data. Across extensive automatic and human evaluations, HalluciDoctor reduces hallucinations by a substantial margin (e.g., up to 44.6% relative) while preserving or improving MLLM performance on perception, cognition, and zero-shot tasks, demonstrating the effectiveness of data-level remediation. The approach emphasizes robustness and generalization, offering a practical path to safer, more reliable multi-modal instruction-following systems and providing publicly available datasets and code for reproducibility.

Abstract

Multi-modal Large Language Models (MLLMs) tuned on machine-generated instruction-following data have demonstrated remarkable performance in various multi-modal understanding and generation tasks. However, the hallucinations inherent in machine-generated data, which could lead to hallucinatory outputs in MLLMs, remain under-explored. This work aims to investigate various hallucinations (i.e., object, relation, attribute hallucinations) and mitigate those hallucinatory toxicities in large-scale machine-generated visual instruction datasets. Drawing on the human ability to identify factual errors, we present a novel hallucination detection and elimination framework, HalluciDoctor, based on the cross-checking paradigm. We use our framework to identify and eliminate hallucinations in the training data automatically. Interestingly, HalluciDoctor also indicates that spurious correlations arising from long-tail object co-occurrences contribute to hallucinations. Based on that, we execute counterfactual visual instruction expansion to balance data distribution, thereby enhancing MLLMs' resistance to hallucinations. Comprehensive experiments on hallucination evaluation benchmarks show that our method successfully mitigates 44.6% hallucinations relatively and maintains competitive performance compared to LLaVA. The data and code for this paper are publicly available. \url{https://github.com/Yuqifan1117/HalluciDoctor}.

Figures (10)

• Figure 1: (a) On the top, we show an example of visual instruction and various hallucinatory toxicities within it. (b) At the bottom, we show that refined LLaVA++ from HalluciDoctor can alleviate hallucinatory toxicity to MLLM and improve its performance.
• Figure 2: Overview of our proposed HalluciDoctor for automatically eliminating hallucinatory toxicity in visual instruction data and enhancing MLLM's resistance to hallucinations. We summarize the process into four steps: (1) HalluciDoctor first extracts description-oriented answers for semantic analysis and formulates corresponding questions. (2) Image-oriented candidate answers for these questions are then gathered from various MLLMs. (3) HalluciDoctor will identify and remove hallucinatory chunks via consistency cross-checking. (4) Lastly, It creates counterfactual instructions guided by preceding steps to expand the dataset and mitigate hallucinations radically.
• Figure 3: Evaluation scores of detailedness and accuracy for descriptions from MiniGPT-4 with different setups. We visualized the total scores using a gray line, where higher scores indicate more detailed descriptions and fewer hallucinations.
• Figure 4: A case study comparing the levels of MLLM hallucination after fine-tuning on various instruction data.
• Figure 5: Quality score (y-axis, higher is better), accuracy score (x-axis, higher is better), and the stability (circle sizes, smaller is better) of MLLMs' responses on OwlEval benchmark.