Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Each Fake News is Fake in its Own Way: An Attribution Multi-Granularity Benchmark for Multimodal Fake News Detection

Hao Guo, Zihan Ma, Zhi Zeng, Minnan Luo, Weixin Zeng, Jiuyang Tang, Xiang Zhao

TL;DR

The paper addresses the gap in multimodal fake news detection by introducing AMG, a dataset with multi-granularity attribution labels (ImageFab, ImageNoE, EntityInc, EventInc, TimeInc) and temporal context across multiple platforms, along with MGCA, a model that jointly detects authenticity and attributes deception using multi-view textual and visual clues. MGCA extracts and aligns clues such as textual and visual entities, time cues, and image events, leveraging CLIP and BERT-based features, PSCC-NET manipulation signals, and Compare-Net coherence modeling to produce both detection and attribution outputs. Extensive experiments demonstrate that AMG is a more challenging benchmark than existing datasets and that MGCA provides strong performance with significant gains in both detection and attribution, while ablations illustrate the critical roles of event-level coherence and temporal consistency. The work advances interpretability and robustness in fake news detection and offers a publicly available resource for future research.

Abstract

Social platforms, while facilitating access to information, have also become saturated with a plethora of fake news, resulting in negative consequences. Automatic multimodal fake news detection is a worthwhile pursuit. Existing multimodal fake news datasets only provide binary labels of real or fake. However, real news is alike, while each fake news is fake in its own way. These datasets fail to reflect the mixed nature of various types of multimodal fake news. To bridge the gap, we construct an attributing multi-granularity multimodal fake news detection dataset \amg, revealing the inherent fake pattern. Furthermore, we propose a multi-granularity clue alignment model \our to achieve multimodal fake news detection and attribution. Experimental results demonstrate that \amg is a challenging dataset, and its attribution setting opens up new avenues for future research.

Each Fake News is Fake in its Own Way: An Attribution Multi-Granularity Benchmark for Multimodal Fake News Detection

TL;DR

The paper addresses the gap in multimodal fake news detection by introducing AMG, a dataset with multi-granularity attribution labels (ImageFab, ImageNoE, EntityInc, EventInc, TimeInc) and temporal context across multiple platforms, along with MGCA, a model that jointly detects authenticity and attributes deception using multi-view textual and visual clues. MGCA extracts and aligns clues such as textual and visual entities, time cues, and image events, leveraging CLIP and BERT-based features, PSCC-NET manipulation signals, and Compare-Net coherence modeling to produce both detection and attribution outputs. Extensive experiments demonstrate that AMG is a more challenging benchmark than existing datasets and that MGCA provides strong performance with significant gains in both detection and attribution, while ablations illustrate the critical roles of event-level coherence and temporal consistency. The work advances interpretability and robustness in fake news detection and offers a publicly available resource for future research.

Abstract

Social platforms, while facilitating access to information, have also become saturated with a plethora of fake news, resulting in negative consequences. Automatic multimodal fake news detection is a worthwhile pursuit. Existing multimodal fake news datasets only provide binary labels of real or fake. However, real news is alike, while each fake news is fake in its own way. These datasets fail to reflect the mixed nature of various types of multimodal fake news. To bridge the gap, we construct an attributing multi-granularity multimodal fake news detection dataset \amg, revealing the inherent fake pattern. Furthermore, we propose a multi-granularity clue alignment model \our to achieve multimodal fake news detection and attribution. Experimental results demonstrate that \amg is a challenging dataset, and its attribution setting opens up new avenues for future research.

Paper Structure

This paper contains 22 sections, 8 equations, 9 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Dataset Construction
  3. Data Collection
  4. Data Processing and Annotation
  5. Data Collation and Analysis
  6. Methodology
  7. Multi-View Clue Collection
  8. Multimodal Feature Learning
  9. Multi-Granularity Clues Alignment
  10. Training and Inference
  11. Experiment
  12. Conclusion
  13. Appendix
  14. Additions to Annotation Process
  15. Examples of Various Category
  16. ...and 7 more sections

Figures (9)

  • Figure 1: Various types of multimodal fake news in Twitter. "Miscaption" means that the caption of the image does not match the text. "Mismatch" indicates the image is related to the text, but from previous similar event. "Image Fabrication" indicates that the image comes from deepfake technology but is not stated.
  • Figure 2: The process of multimodal fake news attribution.
  • Figure 3: Examples of various attributions.
  • Figure 4: Model outline of MGCA.
  • Figure 5: The case study of MGCA in AMG.
  • ...and 4 more figures