MiRAGeNews: Multimodal Realistic AI-Generated News Detection

TL;DR

MiRAGeNews tackles the rising risk of AI-generated fake news by providing a realistic multimodal benchmark of image-caption pairs produced by state-of-the-art diffusion models and GPT-4 captions. The authors introduce MiRAGe, a multimodal detector that fuses image-based and text-based signals using ensembles of linear and concept bottleneck models, achieving strong in-domain performance and robust generalization to unseen generators and publishers. They show that humans struggle with detection (roughly 60% accuracy on images and 54% on captions), underscoring the need for automated tools. The dataset and detector are released to spur progress in robust detection of AI-generated visual and textual misinformation.

Abstract

The proliferation of inflammatory or misleading "fake" news content has become increasingly common in recent years. Simultaneously, it has become easier than ever to use AI tools to generate photorealistic images depicting any scene imaginable. Combining these two -- AI-generated fake news content -- is particularly potent and dangerous. To combat the spread of AI-generated fake news, we propose the MiRAGeNews Dataset, a dataset of 12,500 high-quality real and AI-generated image-caption pairs from state-of-the-art generators. We find that our dataset poses a significant challenge to humans (60% F-1) and state-of-the-art multi-modal LLMs (< 24% F-1). Using our dataset we train a multi-modal detector (MiRAGe) that improves by +5.1% F-1 over state-of-the-art baselines on image-caption pairs from out-of-domain image generators and news publishers. We release our code and data to aid future work on detecting AI-generated content.

Figures (10)

• Figure 1: Multimodal fake news with hyperrealistic generated images from Midjourney poses a significant challenge for both state-of-the-art MLLMs (< 24% F-1) and humans (60% F-1). Our detectors achieve over 98% F-1 on in-domain (ID) data and can generalize on out-of-domain (OOD) data from unseen news publishers and image generators (85% F-1)
• Figure 2: Example of MiRAGeNews dataset generation. We use GPT-4 to generate a misleading caption, which is then used by Midjourney to generate the image.
• Figure 3: Overview of our MiRAGe detector for multimodal AI-generated news detection, which combines MiRAGe-Img with MiRAGe-Txt. MiRAGe-Img trains a linear layer on the outputs from the image linear model and Object-Class Concept Bottleneck Model (CBM), while MiRAGe-Txt trains a linear layer on the outputs from the text linear model and Text Bottleneck Model (TBM). Outputs from two models can be either early fused or late fused to make the final prediction on the image-caption pair.
• Figure 4: (a) Early Fusion detector uses both image and text features together for classification while (b) Late Fusion detector uses outputs from previously trained unimodal detectors.
• Figure 5: We see that MiRAGe-Img outperforms existing image-only detectors in both in-domain (ID) and out-of-domain (OOD). ZS and FT are short for Zero-Shot and Fine-Tuned, respectively