Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Probabilistic Concept Graph Reasoning for Multimodal Misinformation Detection

Ruichao Yang, Wei Gao, Xiaobin Zhu, Jing Ma, Hongzhan Lin, Ziyang Luo, Bo-Wen Zhang, Xu-Cheng Yin

Abstract

Multimodal misinformation poses an escalating challenge that often evades traditional detectors, which are opaque black boxes and fragile against new manipulation tactics. We present Probabilistic Concept Graph Reasoning (PCGR), an interpretable and evolvable framework that reframes multimodal misinformation detection (MMD) as structured and concept-based reasoning. PCGR follows a build-then-infer paradigm, which first constructs a graph of human-understandable concept nodes, including novel high-level concepts automatically discovered and validated by multimodal large language models (MLLMs), and then applies hierarchical attention over this concept graph to infer claim veracity. This design produces interpretable reasoning chains linking evidence to conclusions. Experiments demonstrate that PCGR achieves state-of-the-art MMD accuracy and robustness to emerging manipulation types, outperforming prior methods in both coarse detection and fine-grained manipulation recognition.

Probabilistic Concept Graph Reasoning for Multimodal Misinformation Detection

Abstract

Multimodal misinformation poses an escalating challenge that often evades traditional detectors, which are opaque black boxes and fragile against new manipulation tactics. We present Probabilistic Concept Graph Reasoning (PCGR), an interpretable and evolvable framework that reframes multimodal misinformation detection (MMD) as structured and concept-based reasoning. PCGR follows a build-then-infer paradigm, which first constructs a graph of human-understandable concept nodes, including novel high-level concepts automatically discovered and validated by multimodal large language models (MLLMs), and then applies hierarchical attention over this concept graph to infer claim veracity. This design produces interpretable reasoning chains linking evidence to conclusions. Experiments demonstrate that PCGR achieves state-of-the-art MMD accuracy and robustness to emerging manipulation types, outperforming prior methods in both coarse detection and fine-grained manipulation recognition.

Paper Structure

This paper contains 23 sections, 8 equations, 9 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Formulation
  4. Methodology
  5. Automatic Concept Growth
  6. Text-Image Encoding
  7. Concept Probability Computation
  8. Concept Graph Construction
  9. Multimodal Misinformation Detection
  10. Model Training
  11. Experiments and Results
  12. Experimental Setup
  13. Datasets.
  14. Coarse-level MMD Results
  15. Fine-grained Detection Results
  16. ...and 8 more sections

Figures (9)

  • Figure 1: Human vs. model reasoning on a misinformation example. Left: Conventional multimodal detectors fuse image-text features and may misclassify superficially aligned content (e.g., a fake bombing claim supported by a consistent but unrelated photo). Right: A concept-based reasoning process, used by human fact-checkers and our model, decomposes reasoning into concept-level checks. Each concept yields a soft judgment, collectively leading to the final verdict.
  • Figure 2: The Probabilistic Concept Graph Reasoning (PCGR) framework. PCGR organizes multimodal evidence into interpretable concept nodes and performs hierarchical probabilistic reasoning to infer claim veracity.
  • Figure 3: Statistics of three benchmark datasets.
  • Figure 4: Precision-Recall curves for MMD results on MMFakeBench and AMG (in-domain) and MiRAGeNews (OOD). Blue and green markers denote general-purpose MLLMs and task-specific detectors, respectively. Dashed gray lines indicate iso-F1 contours.
  • Figure 5: Ablation results for coarse-level and fine-grained detection on the AMG dataset.
  • ...and 4 more figures