The Adversarial AI-Art: Understanding, Generation, Detection, and Benchmarking

TL;DR

The paper addresses the security risks posed by AI-generated images and proposes ARIA, a large-scale dataset of real and AI-generated images across five categories. It combines a large user study, extensive detector benchmarking, and a ResNet-50 classifier trained on ARIA to assess human and automated ability to detect AI-art, revealing substantial challenges in both domains. The study shows average human accuracy around 65–68%, with real images easier to identify than AI-generated ones, and detectors often underperform, especially on IT2I content. Overall, ARIA offers a foundation for evaluating defenses against adversarial AI-art and guiding future research in detection, verification, and policy considerations.

Abstract

Generative AI models can produce high-quality images based on text prompts. The generated images often appear indistinguishable from images generated by conventional optical photography devices or created by human artists (i.e., real images). While the outstanding performance of such generative models is generally well received, security concerns arise. For instance, such image generators could be used to facilitate fraud or scam schemes, generate and spread misinformation, or produce fabricated artworks. In this paper, we present a systematic attempt at understanding and detecting AI-generated images (AI-art) in adversarial scenarios. First, we collect and share a dataset of real images and their corresponding artificial counterparts generated by four popular AI image generators. The dataset, named ARIA, contains over 140K images in five categories: artworks (painting), social media images, news photos, disaster scenes, and anime pictures. This dataset can be used as a foundation to support future research on adversarial AI-art. Next, we present a user study that employs the ARIA dataset to evaluate if real-world users can distinguish with or without reference images. In a benchmarking study, we further evaluate if state-of-the-art open-source and commercial AI image detectors can effectively identify the images in the ARIA dataset. Finally, we present a ResNet-50 classifier and evaluate its accuracy and transferability on the ARIA dataset.

Figures (3)

• Figure 1: Sample images from the ARIA dataset.
• Figure 2: The user study: (A) The collection of users' background information; (B) The "training" samples for users with references ($U_R$); (C) The main survey.
• Figure 3: F1-score (%) of ResNet-50 trained and evaluated on different subsets.