ReLoop: "Seeing Twice and Thinking Backwards" via Closed-loop Training to Mitigate Hallucinations in Multimodal understanding

TL;DR

ReLoop introduces a closed-loop training framework for multimodal language models to mitigate hallucinations in open-ended VQA. By freezing three Consistency Feedback Plugins—semantic reconstruction, visual description, and attention supervision—and training a main model with aggregated losses, the method enforces semantic and visual grounding and interpretable attention. The approach demonstrates consistent reductions in hallucinations and improvements in cross-modal faithfulness across multiple backbones and benchmarks, and shows robustness to noisy supervision. This framework offers a practical, scalable path toward more reliable multimodal reasoning and grounding in real-world applications.

Abstract

While Multimodal Large Language Models (MLLMs) have achieved remarkable progress in open-ended visual question answering, they remain vulnerable to hallucinations. These are outputs that contradict or misrepresent input semantics, posing a critical challenge to the reliability and factual consistency. Existing methods often rely on external verification or post-hoc correction, lacking an internal mechanism to validate outputs directly during training. To bridge this gap, we propose ReLoop, a unified closed-loop training framework that encourages multimodal consistency for cross-modal understanding in MLLMs. ReLoop adopts a ring-shaped structure that integrates three complementary consistency feedback mechanisms, obliging MLLMs to "seeing twice and thinking backwards". Specifically, ReLoop employs the frozen Consistency Feedback Plugin (CFP), comprising semantic reconstruction, visual description, and an attention supervision module for attention alignment. These components collectively enforce semantic reversibility, visual consistency, and interpretable attention, enabling the model to correct its outputs during training. Extensive evaluations and analyses demonstrate the effectiveness of ReLoop in reducing hallucination rates across multiple benchmarks, establishing a robust method for hallucination mitigation in MLLMs. We will release our source code and data in the camera-ready version.

Figures (6)

• Figure 1: Illustration of four major hallucination types in open-ended VQA. Despite being visually grounded, MLLMs produce fluent but hallucinated responses across object, attribute, relation, and event dimensions.
• Figure 2: Seeing Twice and Thinking Backwards: ReLooping Hallucination Suppression in Multimodal Language Models. This diagram aligns human cognitive phases (left) with model modules (right) in a closed-loop process. The main model $M$ produces an answer which is then introspected via CFP‑Lang (language reconstruction), CFP‑Vis (visual description), and internal cross‑attention maps. Semantic aggregation, CLIP similarity, and entropy‑based soft masks produce feedback losses that are summed and back‑propagated to update $M$ and the semantic aggregator $S$.
• Figure 3: Token-Level Attention Supervision. Visualization of predicted attention $\mathcal{H}$ and entropy-based pseudo ground truth $\mathcal{H}_{\text{pseudo}}$ for two key answer tokens: dog (top row) and playing (bottom row).
• Figure 4: Type‑wise hallucination rates (%) for baseline (MiniGPT-4) and ReLoop models.
• Figure 5: KDE distributions of CLIP similarity, BERTScore, and attention entropy for hallucinated and non-hallucinated samples. ReLoop's frozen modules exhibit sharp signal shifts that serve as reliable supervision sources.