Multi-modal Misinformation Detection: Approaches, Challenges and Opportunities

TL;DR

This work analyzes, categorizes, and identifies existing approaches in addition to the challenges and shortcomings they face to unearth new research opportunities in the field of multi-modal misinformation detection, and identifies existing approaches in addition to the challenges and shortcomings they face.

Abstract

As social media platforms are evolving from text-based forums into multi-modal environments, the nature of misinformation in social media is also transforming accordingly. Taking advantage of the fact that visual modalities such as images and videos are more favorable and attractive to the users and textual contents are sometimes skimmed carelessly, misinformation spreaders have recently targeted contextual connections between the modalities e.g., text and image. Hence many researchers have developed automatic techniques for detecting possible cross-modal discordance in web-based content. We analyze, categorize and identify existing approaches in addition to challenges and shortcomings they face in order to unearth new research opportunities in the field of multi-modal misinformation detection.

Figures (6)

• Figure 1: An overview of multi-modal misinformation detection pipeline.
• Figure 2: A hybrid of early and late fusion mechanisms.
• Figure 3: An overview of the multi-modal model study.
• Figure 4: Examples of different classes in Fakeddit dataset nakamura-etal-2020-fakeddit.
• Figure 5: Number of news articles by dataset.