WILD: a new in-the-Wild Image Linkage Dataset for synthetic image attribution

TL;DR

WILD addresses the growing need for realistic benchmarks in synthetic image attribution by introducing a two-part dataset: a closed set of $10$ generators and an open set of $10$ generators, each with $1{,}000$ images, plus post-processing to simulate in-the-wild conditions. The authors define careful prompting to link images to prompts in the closed set, and assemble a diverse open set including diffusion and GAN-based methods, to test attribution models beyond training data. They benchmark seven baselines spanning CLIP-based, hybrid multimodal, CNN-based, and Vision Transformer approaches, revealing that while closed-set attribution is relatively easy, robustness to post-processing and open-set generalization remain challenging, with ViT-based methods often showing superior resilience. Overall, WILD provides a rigorous platform for evaluating and developing attribution models capable of operating in realistic, post-processed environments, with implications for copyright, misinformation, and authenticity workflows.

Abstract

Synthetic image source attribution is an open challenge, with an increasing number of image generators being released yearly. The complexity and the sheer number of available generative techniques, as well as the scarcity of high-quality open source datasets of diverse nature for this task, make training and benchmarking synthetic image source attribution models very challenging. WILD is a new in-the-Wild Image Linkage Dataset designed to provide a powerful training and benchmarking tool for synthetic image attribution models. The dataset is built out of a closed set of 10 popular commercial generators, which constitutes the training base of attribution models, and an open set of 10 additional generators, simulating a real-world in-the-wild scenario. Each generator is represented by 1,000 images, for a total of 10,000 images in the closed set and 10,000 images in the open set. Half of the images are post-processed with a wide range of operators. WILD allows benchmarking attribution models in a wide range of tasks, including closed and open set identification and verification, and robust attribution with respect to post-processing and adversarial attacks. Models trained on WILD are expected to benefit from the challenging scenario represented by the dataset itself. Moreover, an assessment of seven baseline methodologies on closed and open set attribution is presented, including robustness tests with respect to post-processing.

Figures (4)

• Figure 1: Two groups of ten images generated by the closed set generators with the same text prompt. The generators are identified by the codes described in Section \ref{['subsec:generators']}. The prompts used to generate the images are: (top, a-j): prompt 439, "Image of an Italian young man who is wearing a pullover. He has a pointy face and full lips. The subject is looking into the camera in happiness. The image is taken with a city in the background"; (bottom, k-t): prompt 266, “A proud British woman in her 20s with hooded brown eyes and a small nose is looking into the camera. She is wearing a sky blue fedora and a necklace. The subject is portrayed with a golf course in the background".
• Figure 2: Some images from the open set section of WILD (one image per generator). The generators are identified by the codes described in Section \ref{['subsec:open_set']}.
• Figure 3: ROC Curves of baselines in the binary classification task between open set (positive class) and closed set (negative task), with different levels of image post-processing: (a) plain images, (b) 1 post-processing step, (c) 2 post-processing steps, (d) 3 post-processing steps.
• Figure 4: Confusion matrices obtained by CLIP+MLP (a), DE-FAKE (b), XceptionNet (c), and VTC (d) on the open set attribution with rejection threshold, without post-processing. The generator-classes are represented by the three-letter codes defined in Section \ref{['subsec:generators']}.