Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MIRAGE: Agentic Framework for Multimodal Misinformation Detection with Web-Grounded Reasoning

Mir Nafis Sharear Shopnil, Sharad Duwal, Abhishek Tyagi, Adiba Mahbub Proma

TL;DR

MIRAGE tackles multimodal misinformation by decomposing verification into four dedicated, model-pluggable modules that separately assess visual authenticity, cross-modal alignment, and retrieval-augmented factual claims, before a calibrated final judgment combines signals with citation-backed rationales. The framework uses external web retrieval to ground claims and outputs transparent JSON evidence, enabling auditing and potential human-in-the-loop workflows. On MMFakeBench, MIRAGE achieves the strongest reported F1 (81.65% on validation) and demonstrates generalization to a 5k test set (81.44% F1), with ablations pinpointing the contributions of visual verification and retrieval-based reasoning. The approach avoids domain-specific training, offering a scalable, interpretable, and cost-conscious path to misinformation detection across modalities where labeled data are scarce, while acknowledging limitations related to niche topics, high-fidelity AI imagery, and deployment costs.

Abstract

Misinformation spreads across web platforms through billions of daily multimodal posts that combine text and images, overwhelming manual fact-checking capacity. Supervised detection models require domain-specific training data and fail to generalize across diverse manipulation tactics. We present MIRAGE, an inference-time, model-pluggable agentic framework that decomposes multimodal verification into four sequential modules: visual veracity assessment detects AI-generated images, cross-modal consistency analysis identifies out-of-context repurposing, retrieval-augmented factual checking grounds claims in web evidence through iterative question generation, and a calibrated judgment module integrates all signals. MIRAGE orchestrates vision-language model reasoning with targeted web retrieval, outputs structured and citation-linked rationales. On MMFakeBench validation set (1,000 samples), MIRAGE with GPT-4o-mini achieves 81.65% F1 and 75.1% accuracy, outperforming the strongest zero-shot baseline (GPT-4V with MMD-Agent at 74.0% F1) by 7.65 points while maintaining 34.3% false positive rate versus 97.3% for a judge-only baseline. Test set results (5,000 samples) confirm generalization with 81.44% F1 and 75.08% accuracy. Ablation studies show visual verification contributes 5.18 F1 points and retrieval-augmented reasoning contributes 2.97 points. Our results demonstrate that decomposed agentic reasoning with web retrieval can match supervised detector performance without domain-specific training, enabling misinformation detection across modalities where labeled data remains scarce.

MIRAGE: Agentic Framework for Multimodal Misinformation Detection with Web-Grounded Reasoning

TL;DR

MIRAGE tackles multimodal misinformation by decomposing verification into four dedicated, model-pluggable modules that separately assess visual authenticity, cross-modal alignment, and retrieval-augmented factual claims, before a calibrated final judgment combines signals with citation-backed rationales. The framework uses external web retrieval to ground claims and outputs transparent JSON evidence, enabling auditing and potential human-in-the-loop workflows. On MMFakeBench, MIRAGE achieves the strongest reported F1 (81.65% on validation) and demonstrates generalization to a 5k test set (81.44% F1), with ablations pinpointing the contributions of visual verification and retrieval-based reasoning. The approach avoids domain-specific training, offering a scalable, interpretable, and cost-conscious path to misinformation detection across modalities where labeled data are scarce, while acknowledging limitations related to niche topics, high-fidelity AI imagery, and deployment costs.

Abstract

Misinformation spreads across web platforms through billions of daily multimodal posts that combine text and images, overwhelming manual fact-checking capacity. Supervised detection models require domain-specific training data and fail to generalize across diverse manipulation tactics. We present MIRAGE, an inference-time, model-pluggable agentic framework that decomposes multimodal verification into four sequential modules: visual veracity assessment detects AI-generated images, cross-modal consistency analysis identifies out-of-context repurposing, retrieval-augmented factual checking grounds claims in web evidence through iterative question generation, and a calibrated judgment module integrates all signals. MIRAGE orchestrates vision-language model reasoning with targeted web retrieval, outputs structured and citation-linked rationales. On MMFakeBench validation set (1,000 samples), MIRAGE with GPT-4o-mini achieves 81.65% F1 and 75.1% accuracy, outperforming the strongest zero-shot baseline (GPT-4V with MMD-Agent at 74.0% F1) by 7.65 points while maintaining 34.3% false positive rate versus 97.3% for a judge-only baseline. Test set results (5,000 samples) confirm generalization with 81.44% F1 and 75.08% accuracy. Ablation studies show visual verification contributes 5.18 F1 points and retrieval-augmented reasoning contributes 2.97 points. Our results demonstrate that decomposed agentic reasoning with web retrieval can match supervised detector performance without domain-specific training, enabling misinformation detection across modalities where labeled data remains scarce.
Paper Structure (45 sections, 2 equations, 1 figure, 3 tables)

This paper contains 45 sections, 2 equations, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Multimodal Misinformation Benchmarks
  4. Vision-Language Models for Fact-Checking
  5. Agentic Reasoning and Tool Use
  6. Visual Forensics and Cross-Modal Consistency
  7. LLM Reliability, Calibration, and Limitations
  8. Methodology
  9. Overview
  10. Visual Verification Module
  11. Relevancy Assessment Module
  12. Claim Verification Module
  13. Question Generation
  14. Web Search and Answer Synthesis
  15. Experimental Setup
  16. ...and 30 more sections

Figures (1)

  • Figure 1: MIRAGE architecture showing the four sequential modules: Visual Verification analyzes images for AI generation and manipulation, Relevancy Assessment evaluates cross-modal alignment, Claim Verification performs retrieval-augmented fact-checking, and Final Judgment integrates all signals to produce calibrated predictions.