DiffSeg30k: A Multi-Turn Diffusion Editing Benchmark for Localized AIGC Detection

TL;DR

DiffSeg30k introduces a 30k-image benchmark of diffusion-edited content with pixel-level masks and explicit editing-model labels, enabling simultaneous localization and attribution of edits across up to three sequential edits from eight diffusion models. A VLM-driven two-stage pipeline automates region detection and context-aware editing prompts, creating a challenging multi-turn localization task that emphasizes generalization to unseen generators. Baseline experiments with FCN-8s, SegFormer, and Deeplabv3+ reveal that model capacity matters and semantic segmentation remains difficult under multi-turn edits, yet segmentation models can outperform traditional AIGC classifiers when repurposed for whole-image classification and generalize across generators. The dataset highlights robustness limitations to common post-processing and motivates further research in robust, transferable diffusion-edit localization methods with practical implications for content verification and digital forensics.

Abstract

Diffusion-based editing enables realistic modification of local image regions, making AI-generated content harder to detect. Existing AIGC detection benchmarks focus on classifying entire images, overlooking the localization of diffusion-based edits. We introduce DiffSeg30k, a publicly available dataset of 30k diffusion-edited images with pixel-level annotations, designed to support fine-grained detection. DiffSeg30k features: 1) In-the-wild images--we collect images or image prompts from COCO to reflect real-world content diversity; 2) Diverse diffusion models--local edits using eight SOTA diffusion models; 3) Multi-turn editing--each image undergoes up to three sequential edits to mimic real-world sequential editing; and 4) Realistic editing scenarios--a vision-language model (VLM)-based pipeline automatically identifies meaningful regions and generates context-aware prompts covering additions, removals, and attribute changes. DiffSeg30k shifts AIGC detection from binary classification to semantic segmentation, enabling simultaneous localization of edits and identification of the editing models. We benchmark three baseline segmentation approaches, revealing significant challenges in semantic segmentation tasks, particularly concerning robustness to image distortions. Experiments also reveal that segmentation models, despite being trained for pixel-level localization, emerge as highly reliable whole-image classifiers of diffusion edits, outperforming established forgery classifiers while showing great potential in cross-generator generalization. We believe DiffSeg30k will advance research in fine-grained localization of AI-generated content by demonstrating the promise and limitations of segmentation-based methods. DiffSeg30k is released at: https://huggingface.co/datasets/Chaos2629/Diffseg30k

Figures (10)

• Figure 1: Motivation. Images shared online may be independently edited multiple times by different users using different models. DiffSeg30k simulates this real-world scenario through multi-turn diffusion-based edits, enabling novel fine-grained AIGC detection - simultaneous edit localization and model attribution.
• Figure 2: Automatic data collection framework. The pipeline consists of two stages: (1) identifying editable regions using VLMs Qwen2.5-VL and Grounded-SAM groundedsam to generate object masks; (2) generating context-appropriate editing prompts with VLMs and applying sequential diffusion-based edits to the selected regions.
• Figure 3: DiffSeg30k Statistics. The distributions reflect our balanced data collection strategy: (a) Object category distribution shows Food (14,308) are most frequently edited, with an overall balanced distribution across categories. (b) Mask area ratio distribution reveals a decreasing trend from smaller to larger regions. (c) Editing count per model indicates uniform participation of all eight diffusion models across first, second, and third editing rounds. (d) We manually increased the frequency of multi-turn editing, with the first, second, and third rounds occurring at an approximate 1:4:5 ratio, as multi-turn editing subsumes single-turn cases and presents a more challenging scenario. (e) The base image ratio is approximately balanced between real and AI-generated images.
• Figure 4: Example edited images. For each image pair, the left shows the original image with red contours marking the regions to be edited, and the right presents the corresponding editing result.
• Figure 5: Confusion matrix on the semantic segmentation task.