Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Comprehensive Dataset for Human vs. AI Generated Image Detection

Rajarshi Roy, Nasrin Imanpour, Ashhar Aziz, Shashwat Bajpai, Gurpreet Singh, Shwetangshu Biswas, Kapil Wanaskar, Parth Patwa, Subhankar Ghosh, Shreyas Dixit, Nilesh Ranjan Pal, Vipula Rawte, Ritvik Garimella, Gaytri Jena, Vasu Sharma, Vinija Jain, Aman Chadha, Aishwarya Naresh Reganti, Amitava Das

TL;DR

The paper tackles the rising challenge of distinguishing AI-generated images from real photographs and attributing them to specific generators. It introduces MS COCOAI, a large-scale, caption-aligned dataset built on MS COCO, containing 96,000 image-caption pairs generated across five models (Stable Diffusion 3, Stable Diffusion 2.1, SDXL, DALL-E 3, MidJourney v6) plus 16,000 real samples, with model provenance and perturbations. Two tasks are defined: binary real-vs-generated classification and model attribution among the five generators; a frequency-domain baseline using a $2$D Fourier Transform fed into ResNet-50 yields Task A accuracy of around 0.80 and Task B accuracy around 0.45, highlighting the relative difficulty of attribution. The dataset enables robust evaluation under perturbations and supports cross-model fingerprinting and cross-modal learning, having practical impact for misinformation mitigation and digital forensics.

Abstract

Multimodal generative AI systems like Stable Diffusion, DALL-E, and MidJourney have fundamentally changed how synthetic images are created. These tools drive innovation but also enable the spread of misleading content, false information, and manipulated media. As generated images become harder to distinguish from photographs, detecting them has become an urgent priority. To combat this challenge, We release MS COCOAI, a novel dataset for AI generated image detection consisting of 96000 real and synthetic datapoints, built using the MS COCO dataset. To generate synthetic images, we use five generators: Stable Diffusion 3, Stable Diffusion 2.1, SDXL, DALL-E 3, and MidJourney v6. Based on the dataset, we propose two tasks: (1) classifying images as real or generated, and (2) identifying which model produced a given synthetic image. The dataset is available at https://huggingface.co/datasets/Rajarshi-Roy-research/Defactify_Image_Dataset.

A Comprehensive Dataset for Human vs. AI Generated Image Detection

TL;DR

The paper tackles the rising challenge of distinguishing AI-generated images from real photographs and attributing them to specific generators. It introduces MS COCOAI, a large-scale, caption-aligned dataset built on MS COCO, containing 96,000 image-caption pairs generated across five models (Stable Diffusion 3, Stable Diffusion 2.1, SDXL, DALL-E 3, MidJourney v6) plus 16,000 real samples, with model provenance and perturbations. Two tasks are defined: binary real-vs-generated classification and model attribution among the five generators; a frequency-domain baseline using a D Fourier Transform fed into ResNet-50 yields Task A accuracy of around 0.80 and Task B accuracy around 0.45, highlighting the relative difficulty of attribution. The dataset enables robust evaluation under perturbations and supports cross-model fingerprinting and cross-modal learning, having practical impact for misinformation mitigation and digital forensics.

Abstract

Multimodal generative AI systems like Stable Diffusion, DALL-E, and MidJourney have fundamentally changed how synthetic images are created. These tools drive innovation but also enable the spread of misleading content, false information, and manipulated media. As generated images become harder to distinguish from photographs, detecting them has become an urgent priority. To combat this challenge, We release MS COCOAI, a novel dataset for AI generated image detection consisting of 96000 real and synthetic datapoints, built using the MS COCO dataset. To generate synthetic images, we use five generators: Stable Diffusion 3, Stable Diffusion 2.1, SDXL, DALL-E 3, and MidJourney v6. Based on the dataset, we propose two tasks: (1) classifying images as real or generated, and (2) identifying which model produced a given synthetic image. The dataset is available at https://huggingface.co/datasets/Rajarshi-Roy-research/Defactify_Image_Dataset.
Paper Structure (13 sections, 3 figures, 3 tables)

This paper contains 13 sections, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. AI-Generated Image Datasets
  4. Detection Methods
  5. Dataset
  6. Image generation and Annotation
  7. Perturbations
  8. Data Structure
  9. Data Analysis
  10. CT2 - AI Generated Image Detection Tasks
  11. Baseline
  12. Results
  13. Conclusion

Figures (3)

  • Figure 1: Images generated from the same caption. Each model produces visually distinct outputs, highlighting the challenge of AI-generated image detection.
  • Figure 2: Word cloud visualization of all captions in the dataset. Word size corresponds to term frequency, revealing the semantic distribution across the corpus. Dominant terms reflect a comprehensive coverage of everyday scenes, common objects, animals, human activities, and color descriptors.
  • Figure 3: Baseline workflow. The input image is first transformed into its frequency domain representation and then passed through a ResNet-50 CNN classifier to predict whether it is real or fake.