Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ExDR: Explanation-driven Dynamic Retrieval Enhancement for Multimodal Fake News Detection

Guoxuan Ding, Yuqing Li, Ziyan Zhou, Zheng Lin, Daren Zha, Jiangnan Li

TL;DR

ExDR introduces an explanation-driven dynamic retrieval framework for multimodal fake news detection, addressing when to retrieve, how to retrieve, and what to retrieve. The method leverages model-generated explanations to build three-level confidence signals for triggering retrieval, and employs entity-enriched hybrid indexing with contrastive evidence to improve retrieval quality. Experiments on AMG and MR$^2$ show ExDR outperforms baselines in retrieval triggering, evidence quality, and overall detection accuracy, with robust cross-domain generalization. The approach offers a practical, efficient path to evidence-aware MFND, enabling more reliable misinformation detection in dynamic information environments.

Abstract

The rapid spread of multimodal fake news poses a serious societal threat, as its evolving nature and reliance on timely factual details challenge existing detection methods. Dynamic Retrieval-Augmented Generation provides a promising solution by triggering keyword-based retrieval and incorporating external knowledge, thus enabling both efficient and accurate evidence selection. However, it still faces challenges in addressing issues such as redundant retrieval, coarse similarity, and irrelevant evidence when applied to deceptive content. In this paper, we propose ExDR, an Explanation-driven Dynamic Retrieval-Augmented Generation framework for Multimodal Fake News Detection. Our framework systematically leverages model-generated explanations in both the retrieval triggering and evidence retrieval modules. It assesses triggering confidence from three complementary dimensions, constructs entity-aware indices by fusing deceptive entities, and retrieves contrastive evidence based on deception-specific features to challenge the initial claim and enhance the final prediction. Experiments on two benchmark datasets, AMG and MR2, demonstrate that ExDR consistently outperforms previous methods in retrieval triggering accuracy, retrieval quality, and overall detection performance, highlighting its effectiveness and generalization capability.

ExDR: Explanation-driven Dynamic Retrieval Enhancement for Multimodal Fake News Detection

TL;DR

ExDR introduces an explanation-driven dynamic retrieval framework for multimodal fake news detection, addressing when to retrieve, how to retrieve, and what to retrieve. The method leverages model-generated explanations to build three-level confidence signals for triggering retrieval, and employs entity-enriched hybrid indexing with contrastive evidence to improve retrieval quality. Experiments on AMG and MR show ExDR outperforms baselines in retrieval triggering, evidence quality, and overall detection accuracy, with robust cross-domain generalization. The approach offers a practical, efficient path to evidence-aware MFND, enabling more reliable misinformation detection in dynamic information environments.

Abstract

The rapid spread of multimodal fake news poses a serious societal threat, as its evolving nature and reliance on timely factual details challenge existing detection methods. Dynamic Retrieval-Augmented Generation provides a promising solution by triggering keyword-based retrieval and incorporating external knowledge, thus enabling both efficient and accurate evidence selection. However, it still faces challenges in addressing issues such as redundant retrieval, coarse similarity, and irrelevant evidence when applied to deceptive content. In this paper, we propose ExDR, an Explanation-driven Dynamic Retrieval-Augmented Generation framework for Multimodal Fake News Detection. Our framework systematically leverages model-generated explanations in both the retrieval triggering and evidence retrieval modules. It assesses triggering confidence from three complementary dimensions, constructs entity-aware indices by fusing deceptive entities, and retrieves contrastive evidence based on deception-specific features to challenge the initial claim and enhance the final prediction. Experiments on two benchmark datasets, AMG and MR2, demonstrate that ExDR consistently outperforms previous methods in retrieval triggering accuracy, retrieval quality, and overall detection performance, highlighting its effectiveness and generalization capability.
Paper Structure (34 sections, 18 equations, 6 figures, 7 tables)

This paper contains 34 sections, 18 equations, 6 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Preliminaries
  5. Task Definition
  6. Data Preparation
  7. Response Formation
  8. When to Retrieve: Multi-Level Confidence Estimation
  9. Label-level Uncertainty.
  10. Token-level Support.
  11. Sentence-level Confidence.
  12. Confidence Threshold Optimization
  13. How to Retrieve: Entity-enriched Multimodal Hybrid Indexing
  14. What to Retrieve: Multi-Granularity Deceptive Evidence
  15. Explanation-Guided Label Inference
  16. ...and 19 more sections

Figures (6)

  • Figure 1: Overview of our proposed ExDR framework. ExDR consists of two main components: (1) a retrieval triggering module that dynamically determines whether retrieval is necessary based on response analysis, and (2) an evidence retrieval module that retrieves targeted evidence, including both positive and negative samples guided by fine-grained deception labels, to enrich the context and improve model generation.
  • Figure 2: Case study illustrating the effectiveness of ExDR across three representative scenarios.
  • Figure 3: AMG - Vanilla
  • Figure 4: MR$^2$ - Vanilla
  • Figure 5: AMG - Fine-tuned
  • ...and 1 more figures