Seeing is Believing: Rich-Context Hallucination Detection for MLLMs via Backward Visual Grounding

TL;DR

This work tackles visual hallucination in multimodal LLMs by introducing VBackChecker, a reference-free detector that grounds each rich-context sentence back to the input image. It combines a pixel-level grounding LLM with reasoning and a mask decoder, producing [SEG] or [REJ] outputs and enabling interpretable explanations. A novel data pipeline, R-Instruct, generates rich-context instruction-tuning data with grounding masks and hard negatives, while R$^2$-HalBench provides a real-world, diverse benchmark across 18 MLLMs to assess both grounding and hallucination detection. Experimental results show state-of-the-art performance on R$^2$-HalBench, strong pixel-level grounding gains, and competitive results relative to GPT-4o, demonstrating practical potential for reliable multimodal systems.

Abstract

Multimodal Large Language Models (MLLMs) have unlocked powerful cross-modal capabilities, but still significantly suffer from hallucinations. As such, accurate detection of hallucinations in MLLMs is imperative for ensuring their reliability in practical applications. To this end, guided by the principle of "Seeing is Believing", we introduce VBackChecker, a novel reference-free hallucination detection framework that verifies the consistency of MLLMgenerated responses with visual inputs, by leveraging a pixellevel Grounding LLM equipped with reasoning and referring segmentation capabilities. This reference-free framework not only effectively handles rich-context scenarios, but also offers interpretability. To facilitate this, an innovative pipeline is accordingly designed for generating instruction-tuning data (R-Instruct), featuring rich-context descriptions, grounding masks, and hard negative samples. We further establish R^2 -HalBench, a new hallucination benchmark for MLLMs, which, unlike previous benchmarks, encompasses real-world, rich-context descriptions from 18 MLLMs with high-quality annotations, spanning diverse object-, attribute, and relationship-level details. VBackChecker outperforms prior complex frameworks and achieves state-of-the-art performance on R^2 -HalBench, even rivaling GPT-4o's capabilities in hallucination detection. It also surpasses prior methods in the pixel-level grounding task, achieving over a 10% improvement. All codes, data, and models are available at https://github.com/PinxueGuo/VBackChecker.

Figures (6)

• Figure 1: Illustration of our proposed VBackChecker. Unlike previous methods (Top), our VBackChecker (Bottom) identifies hallucination in MLLMs by leveraging a pixel-level grounding approach for backward verification, without any references or external experts.
• Figure 2: The framework of our proposed VBackChecker
• Figure 3: The Generation Pipeline of Rich-Context Instruct Tuning Data ( R-Instruct).
• Figure 4: A comprehensive analysis of our R$^2$-HalBench across four distinct dimensions.
• Figure 5: Samples of our R$^2$-HalBench. Each rich-context response, generated from various advanced MLLMs, is systematically annotated with corresponding hallucination categories.