Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Agentic Mixed-Source Multi-Modal Misinformation Detection with Adaptive Test-Time Scaling

Wei Jiang, Tong Chen, Wei Yuan, Quoc Viet Hung Nguyen, Hongzhi Yin

TL;DR

An adaptive test-time scaling paradigm in which each modality-specific VLM agent applies a Best-of-N mechanism, coupled with a critic agent for task-aligned scoring is introduced, to amplify the reasoning capability of VLMs.

Abstract

Vision-language models (VLMs) have been proven effective for detecting multi-modal misinformation on social platforms, especially in zero-shot settings with unavailable or delayed annotations. However, a single VLM's capacity falls short in the more complex mixed-source multi-modal misinformation detection (M3D) task. Taking captioned images as an example, in M3D, false information can originate from untruthful texts, forged images, or mismatches between the two modalities. Although recent agentic systems can handle zero-shot M3D by connecting modality-specific VLM agents, their effectiveness is still bottlenecked by their architecture. In existing agentic M3D solutions, for any input sample, each agent performs only one forward reasoning pass, making decisions prone to model randomness and reasoning errors in challenging cases. Moreover, the lack of exploration over alternative reasoning paths prevents modern VLMs from fully utilizing their reasoning capacity. In this work, we present AgentM3D, a multi-agent framework for zero-shot M3D. To amplify the reasoning capability of VLMs, we introduce an adaptive test-time scaling paradigm in which each modality-specific VLM agent applies a Best-of-N mechanism, coupled with a critic agent for task-aligned scoring. The agents are organized in a cascading, modality-specific decision chain to reduce unnecessary computation and limit error propagation. To ensure scalability, a planning agent dynamically determines the maximum number of reasoning paths based on sample difficulty, and an adaptive stopping mechanism prevents excessive reasoning within each agent. Extensive experiments on two M3D benchmarks demonstrate that AgentM3D achieves state-of-the-art zero-shot detection performance compared with various VLM-based and agentic baselines.

Agentic Mixed-Source Multi-Modal Misinformation Detection with Adaptive Test-Time Scaling

TL;DR

An adaptive test-time scaling paradigm in which each modality-specific VLM agent applies a Best-of-N mechanism, coupled with a critic agent for task-aligned scoring is introduced, to amplify the reasoning capability of VLMs.

Abstract

Vision-language models (VLMs) have been proven effective for detecting multi-modal misinformation on social platforms, especially in zero-shot settings with unavailable or delayed annotations. However, a single VLM's capacity falls short in the more complex mixed-source multi-modal misinformation detection (M3D) task. Taking captioned images as an example, in M3D, false information can originate from untruthful texts, forged images, or mismatches between the two modalities. Although recent agentic systems can handle zero-shot M3D by connecting modality-specific VLM agents, their effectiveness is still bottlenecked by their architecture. In existing agentic M3D solutions, for any input sample, each agent performs only one forward reasoning pass, making decisions prone to model randomness and reasoning errors in challenging cases. Moreover, the lack of exploration over alternative reasoning paths prevents modern VLMs from fully utilizing their reasoning capacity. In this work, we present AgentM3D, a multi-agent framework for zero-shot M3D. To amplify the reasoning capability of VLMs, we introduce an adaptive test-time scaling paradigm in which each modality-specific VLM agent applies a Best-of-N mechanism, coupled with a critic agent for task-aligned scoring. The agents are organized in a cascading, modality-specific decision chain to reduce unnecessary computation and limit error propagation. To ensure scalability, a planning agent dynamically determines the maximum number of reasoning paths based on sample difficulty, and an adaptive stopping mechanism prevents excessive reasoning within each agent. Extensive experiments on two M3D benchmarks demonstrate that AgentM3D achieves state-of-the-art zero-shot detection performance compared with various VLM-based and agentic baselines.
Paper Structure (24 sections, 19 equations, 7 figures, 1 table, 1 algorithm)

This paper contains 24 sections, 19 equations, 7 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Methodology
  4. Hierarchical Modality-specific Detection
  5. Best-of-N Reasoning with Critique-aware Ranking
  6. Adaptive Test-time Scaling Planning
  7. Probabilistic Interpretation of Detection Agent Inference
  8. Experiments
  9. Experimental Settings
  10. Performance Comparison (RQ1)
  11. Analysis of Inference Efficiency (RQ2)
  12. Ablation Study (RQ3)
  13. Hyperparameter Analysis (RQ4)
  14. Impact of the Hyperparameter $\tau$ in Early-Stopping Ranking
  15. Impact of $N$ in BoN Reasoning
  16. ...and 9 more sections

Figures (7)

  • Figure 1: Comparison between single-source and mixed-source multi-modal misinformation detection.
  • Figure 2: The overall structure of AgentM$^3$D. A planning agent routes each input to either standard reasoning or critique-aware Best-of-$N$ reasoning. The latter explores multiple reasoning trajectories, integrates reward and critique signals for candidate selection, and applies adaptive early-stopping ranking to dynamically terminate inference when confident scores emerge.
  • Figure 3: Inference efficiency comparison between AgentM$^3$D and baseline methods.
  • Figure 4: Impact of key components of AgentM$^3$D.
  • Figure 5: Effect of the early-stopping threshold $\tau$ on detection accuracy and early-stopped sample ratio.
  • ...and 2 more figures