Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Stage Verification-Centric Framework for Mitigating Hallucination in Multi-Modal RAG

Baiyu Chen, Wilson Wongso, Xiaoqian Hu, Yue Tan, Flora Salim

TL;DR

This work addresses hallucinations in multimodal, multi-turn RAG by proposing a verification-centric pipeline that combines lightweight query routing, query-aware retrieval with dynamic filtering, dual-path generation, and a Chain-of-Verification finalization. The approach emphasizes factual grounding and reliability over completeness, and is validated on the CRAG-MM benchmark where ablations demonstrate the value of verification stages in reducing hallucinations. Key innovations include a MAD-based dynamic threshold for retrieval pruning, a dual-path generation with self-consistency checks, and a structured verification workflow that decomposes queries into sub-questions for granular validation. The results, along with a public implementation, offer practical guidance for deploying reliable MM-RAG in real-world, egocentric AR/XR and smart assistant contexts.

Abstract

This paper presents the technical solution developed by team CRUISE for the KDD Cup 2025 Meta Comprehensive RAG Benchmark for Multi-modal, Multi-turn (CRAG-MM) challenge. The challenge aims to address a critical limitation of modern Vision Language Models (VLMs): their propensity to hallucinate, especially when faced with egocentric imagery, long-tail entities, and complex, multi-hop questions. This issue is particularly problematic in real-world applications where users pose fact-seeking queries that demand high factual accuracy across diverse modalities. To tackle this, we propose a robust, multi-stage framework that prioritizes factual accuracy and truthfulness over completeness. Our solution integrates a lightweight query router for efficiency, a query-aware retrieval and summarization pipeline, a dual-pathways generation and a post-hoc verification. This conservative strategy is designed to minimize hallucinations, which incur a severe penalty in the competition's scoring metric. Our approach achieved 3rd place in Task 1, demonstrating the effectiveness of prioritizing answer reliability in complex multi-modal RAG systems. Our implementation is available at https://github.com/Breezelled/KDD-Cup-2025-Meta-CRAG-MM .

Multi-Stage Verification-Centric Framework for Mitigating Hallucination in Multi-Modal RAG

TL;DR

This work addresses hallucinations in multimodal, multi-turn RAG by proposing a verification-centric pipeline that combines lightweight query routing, query-aware retrieval with dynamic filtering, dual-path generation, and a Chain-of-Verification finalization. The approach emphasizes factual grounding and reliability over completeness, and is validated on the CRAG-MM benchmark where ablations demonstrate the value of verification stages in reducing hallucinations. Key innovations include a MAD-based dynamic threshold for retrieval pruning, a dual-path generation with self-consistency checks, and a structured verification workflow that decomposes queries into sub-questions for granular validation. The results, along with a public implementation, offer practical guidance for deploying reliable MM-RAG in real-world, egocentric AR/XR and smart assistant contexts.

Abstract

This paper presents the technical solution developed by team CRUISE for the KDD Cup 2025 Meta Comprehensive RAG Benchmark for Multi-modal, Multi-turn (CRAG-MM) challenge. The challenge aims to address a critical limitation of modern Vision Language Models (VLMs): their propensity to hallucinate, especially when faced with egocentric imagery, long-tail entities, and complex, multi-hop questions. This issue is particularly problematic in real-world applications where users pose fact-seeking queries that demand high factual accuracy across diverse modalities. To tackle this, we propose a robust, multi-stage framework that prioritizes factual accuracy and truthfulness over completeness. Our solution integrates a lightweight query router for efficiency, a query-aware retrieval and summarization pipeline, a dual-pathways generation and a post-hoc verification. This conservative strategy is designed to minimize hallucinations, which incur a severe penalty in the competition's scoring metric. Our approach achieved 3rd place in Task 1, demonstrating the effectiveness of prioritizing answer reliability in complex multi-modal RAG systems. Our implementation is available at https://github.com/Breezelled/KDD-Cup-2025-Meta-CRAG-MM .

Paper Structure

This paper contains 20 sections, 8 equations, 2 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Benchmark and Task Definition
  3. Benchmark Overview
  4. Task Definition
  5. Evaluation
  6. Methodology
  7. Lightweight Query Routing
  8. Query-Aware Retrieval
  9. Dual-Path Generation
  10. Verification and Finalization
  11. Experiments
  12. Metrics
  13. Single-source Augmentation
  14. Limitation
  15. Conclusion
  16. ...and 5 more sections

Figures (2)

  • Figure 1: The overall pipeline of our solution.
  • Figure 2: The detail of Query-Aware Retrieval module.