InEx: Hallucination Mitigation via Introspection and Cross-Modal Multi-Agent Collaboration

TL;DR

This work tackles hallucination in multimodal LLMs by introducing InEx, a training-free framework that blends internal introspective reasoning guided by TVER with external cross-modal collaboration involving textual and visual self-reflection and an image-editing agent. The approach iteratively verifies and refines responses until cross-modal consensus is achieved, underpinned by theoretical results linking information flow and the information bottleneck. Empirical results across hallucination and general-purpose benchmarks show that InEx consistently outperforms strong baselines, with robust ablations and significance testing supporting the contributions. The method demonstrates a path toward autonomous, reliable multimodal reasoning without model retraining, with potential extension to additional modalities.

Abstract

Hallucination remains a critical challenge in large language models (LLMs), hindering the development of reliable multimodal LLMs (MLLMs). Existing solutions often rely on human intervention or underutilize the agent's ability to autonomously mitigate hallucination. To address these limitations, we draw inspiration from how humans make reliable decisions in the real world. They begin with introspective reasoning to reduce uncertainty and form an initial judgment, then rely on external verification from diverse perspectives to reach a final decision. Motivated by this cognitive paradigm, we propose InEx, a training-free, multi-agent framework designed to autonomously mitigate hallucination. InEx introduces internal introspective reasoning, guided by entropy-based uncertainty estimation, to improve the reliability of the decision agent's reasoning process. The agent first generates a response, which is then iteratively verified and refined through external cross-modal multi-agent collaboration with the editing agent and self-reflection agents, further enhancing reliability and mitigating hallucination. Extensive experiments show that InEx consistently outperforms existing methods, achieving 4%-27% gains on general and hallucination benchmarks, and demonstrating strong robustness.

Figures (5)

• Figure 1: HTML]F5D6E3 Hallucination HTML]C2D6EC No Hallucination Uncertainty calibration across different methods, with each column showing uncertainty score distributions (top) and corresponding hallucination rates (bottom) across bins.
• Figure 2: Overview of InEx. In: A decision agent initiates introspective reasoning guided by unsupervised uncertainty estimation, producing an initial response grounded in internal uncertainty signals. Ex: The response is then iteratively refined through alternating cross-modal verification and introspective updates, where self-reflection agents assess consistency with visual and textual evidence, and the decision agent continues to revise its output accordingly.
• Figure 3: InEx Parameter Analysis on MMBench using LLaVA-1.5-7B.
• Figure 4: We conduct an ablation study to analyze the effect of 4 different image editing models using LLaVa-1.5-7B.
• Figure 5: Case study on LLaVA-Bench comparing responses from standard decoding and our method using LLaVA-1.5. GPT-4V-aided evaluations are shown, with hallucinated and accurate content highlighted in red and blue.

Theorems & Definitions (3)

• Theorem 1
• Theorem 2
• Theorem 3