Fact or Fake? Assessing the Role of Deepfake Detectors in Multimodal Misinformation Detection

TL;DR

This work investigates whether image-based deepfake detectors meaningfully aid multimodal misinformation verification. It contrasts three system families: image-only detectors, an evidence-centric fact-checking pipeline using goal-directed evidence acquisition and multi-agent debate, and a detector-augmented hybrid. Across MMFakeBench and DGM4, detectors show limited standalone value and can degrade performance when injected as authenticity priors; the evidence-centric approach consistently achieves the strongest results, highlighting the primacy of semantic content and external evidence in image–text claim verification. The findings advocate for evidence-grounded verification and caution against over-reliance on pixel-level artifacts in real-world multimodal misinformation pipelines.

Abstract

In multimodal misinformation, deception usually arises not just from pixel-level manipulations in an image, but from the semantic and contextual claim jointly expressed by the image-text pair. Yet most deepfake detectors, engineered to detect pixel-level forgeries, do not account for claim-level meaning, despite their growing integration in automated fact-checking (AFC) pipelines. This raises a central scientific and practical question: Do pixel-level detectors contribute useful signal for verifying image-text claims, or do they instead introduce misleading authenticity priors that undermine evidence-based reasoning? We provide the first systematic analysis of deepfake detectors in the context of multimodal misinformation detection. Using two complementary benchmarks, MMFakeBench and DGM4, we evaluate: (1) state-of-the-art image-only deepfake detectors, (2) an evidence-driven fact-checking system that performs tool-guided retrieval via Monte Carlo Tree Search (MCTS) and engages in deliberative inference through Multi-Agent Debate (MAD), and (3) a hybrid fact-checking system that injects detector outputs as auxiliary evidence. Results across both benchmark datasets show that deepfake detectors offer limited standalone value, achieving F1 scores in the range of 0.26-0.53 on MMFakeBench and 0.33-0.49 on DGM4, and that incorporating their predictions into fact-checking pipelines consistently reduces performance by 0.04-0.08 F1 due to non-causal authenticity assumptions. In contrast, the evidence-centric fact-checking system achieves the highest performance, reaching F1 scores of approximately 0.81 on MMFakeBench and 0.55 on DGM4. Overall, our findings demonstrate that multimodal claim verification is driven primarily by semantic understanding and external evidence, and that pixel-level artifact signals do not reliably enhance reasoning over real-world image-text misinformation.

Figures (7)

• Figure 1: Multimodal misinformation often uses authentic or synthetic images paired with false narratives. We compare (1) image-only deepfake detectors, (2) an evidence-centric fact-checking system, and (3) a hybrid that incorporates detector outputs, to ask whether pixel-level authenticity cues help verify image–text claims.
• Figure 2: Overview of the proposed evidence-centric fact-checking framework. Given an image--text claim, the system performs verification in two stages. (1) Multi-source verification via MCTS: a multimodal LLM plans tool calls across an extensible toolkit (e.g., web/RAG search, image understanding, counterfactual/VQA, entity and time detection), executes these actions within an environment, and evaluates each trajectory using utility and confidence scores to generate text-side and image-side stances with their corresponding evidence packs. (2) Multi-Agent Debate (MAD): skeptic and supporter agents deliberate over the collected evidence, while a neutral Judge LLM aggregates their arguments to issue the final verdict.
• Figure 3: The confusion matrix of the fact-checking with and without image-based deepfake detector on MMFakebench dataset.
• Figure 4: Confusion matrices on the DGM$^{4}$ dataset for the evidence-centric fact-checking system and the hybrid system integrating a deepfake detector.
• Figure 5: Qualitative cases showing robust evidence-centric verification and a failure mode of hybrid fact-checking caused by misleading deepfake detector outputs.