Seeing Through Deception: Uncovering Misleading Creator Intent in Multimodal News with Vision-Language Models

TL;DR

This work targets the challenge of detecting misleading creator intent in multimodal news by introducing DeceptionDecoded, a $12{,}000$-instance benchmark of image–caption–article triplets $(I, T, A)$ anchored to trustworthy contexts. It formalizes creator intent along two dimensions (desired influence and execution plan) and uses an intent-guided synthesis pipeline to produce both misleading and non-misleading variants across six categories, enabling three tasks: Misleading Intent Detection, Misleading Source Attribution, and Creator Desire Inference. Evaluations of 14 vision-language models reveal that current systems struggle with intent reasoning, relying on surface cues and cross-modal consistency, though consistency-oriented approaches perform better and fine-tuning transfers to external MMD benchmarks. The study underscores the need for authentic, intent-aware models and provides a principled benchmark and methodology to guide future research and governance of multimodal misinformation at scale, including the growing realism of generated images and its implications for detection.

Abstract

The impact of misinformation arises not only from factual inaccuracies but also from the misleading narratives that creators deliberately embed. Interpreting such creator intent is therefore essential for multimodal misinformation detection (MMD) and effective information governance. To this end, we introduce DeceptionDecoded, a large-scale benchmark of 12,000 image-caption pairs grounded in trustworthy reference articles, created using an intent-guided simulation framework that models both the desired influence and the execution plan of news creators. The dataset captures both misleading and non-misleading cases, spanning manipulations across visual and textual modalities, and supports three intent-centric tasks: (1) misleading intent detection, (2) misleading source attribution, and (3) creator desire inference. We evaluate 14 state-of-the-art vision-language models (VLMs) and find that they struggle with intent reasoning, often relying on shallow cues such as surface-level alignment, stylistic polish, or heuristic authenticity signals. These results highlight the limitations of current VLMs and position DeceptionDecoded as a foundation for developing intent-aware models that go beyond shallow cues in MMD.

Figures (11)

• Figure 1: Why creator intent detection matters in multimodal news. Creators can distort trustworthy news contexts by crafting intent-loaded misinformation that appears semantically aligned (e.g., portraying broken icebergs) yet deliberately conveys false narratives (e.g., attributing them to secret underwater nuclear tests).
• Figure 2: DeceptionDecoded: Overview of multimodal news curation guided by diverse simulated creator intents, covering both misleading and non-misleading cases.
• Figure 3: Misleading intent detection performance of VLMs on misleading text presented either in a professional tone (as used in the original DeceptionDecoded setting) versus an explicitly misleading style.
• Figure 4: Case Study: a VLM (GPT-4o-mini) fails to detect misleading creator intent in a FLUX-generated image. The model overlooks the presence of a crowd with fireworks, an unsubstantiated addition given the powerhouse fire described in both the news context and the caption. In contrast, the state-of-the-art GPT-image-1 model produces a more vivid depiction of fireworks, enabling the correct prediction.
• Figure 5: Prompt for selecting VisualNews liu2021visual samples as source data for DeceptionDecoded.