Expertise elevates AI usage: experimental evidence comparing laypeople and professional artists

TL;DR

This study investigates whether professional visual-art expertise transfers to using generative AI for image creation by comparing 50 artists, 49 matched laypeople, and GPT-4o on copying and divergent-image tasks using a text-to-image model. It employs a preregistered, two-task design with prompting and curation across eight trials, quantified via CLIP-based cosine similarity of embeddings. Key findings show artists are more accurate at copying and more divergent in creative tasks than laypeople, while GPT-4o often matches or surpasses average artist performance but does not beat the top humans; curation differences are modest. The results underscore the value of integrating artistic skills with AI tools, suggesting collaborative intelligence between humans and AI with implications for art education and creative industries, while also outlining limitations and directions for broader future work.

Abstract

Generative AI's novel capacities raise questions about the future role of human expertise: does AI level the playing field between professional artists and laypeople, or does expertise enhance AI use? Do the cognitive skills experts make use of in analyzing and drawing visual art also transfer to using these new tools? This pre-registered study conducts experimental comparisons between 50 professional artists and a demographically matched sample of laypeople. Our interdisciplinary team developed two tasks involving image replication and creative image creation, assessing their copying accuracy and divergent thinking. We implemented a bespoke platform for the experiment, powered by a modern text-to-image AI. Results reveal artists produced more accurate copies and more divergent ideas than lay participants, highlighting a skill transfer of professional expertise - even to the confined space of generative AI. We also explored how well an exemplary vision-capable large language model (GPT-4o) would fare: on par in copying and slightly better on average than artists in the creative task, although never above best humans. These findings highlight the importance of integrating artistic skills with AI, suggesting a potential for collaborative synergy that could reshape creative industries and arts education.

Figures (11)

• Figure 1: Motivation for the study. Left side: Generative AI can be compared to other technological developments that have shifted how visual art was made in the past, such as the industrial production of paint or the invention of photography. Right side: The traditional role of the expert directly producing art (top arrow) is now challenged by both laypeople and AI systems producing art via generative AI intermediaries. However, the skills of experts might also transfer to using generative AI themselves (second arrow from the top), opening up new opportunities as well.
• Figure 2: Pipeline overview. The participants were asked to view a reference image and write a prompt for a generative model to either create a similar image (copying task) or a maximally different image (creative task). In the curation phase, they were shown four generated variants and asked to select the most suitable one given the goal of the task. We later compared the similarity of the reference and generated images using an image embedding model, and used cosine vector similarity to operationalize the results. The prompts and variants are from one of the artist participants; the rest of the images created by this participant can be found in the Appendix.
• Figure 3: Experimental results: artists are better at using generative AI than laypeople. The vertical axis is cosine similarity; for the copying task, the goal was to produce similar images (high values), and for creative, dissimilar. The panels display three views of the data. Each dot in (A) represents the distance of a generated image from the reference (averaged across its four variants), the central unit of analysis. The columns of dots are organized by the four stimuli in the two tasks. Some images were harder than others, e.g. the "couple" image was most difficult to copy for all groups (rightmost dot column in Copying panel; cf. images in Figure \ref{['figure_examplegrid']}; see the SI for an extended comparison graph). The darker error bars show stimuli means with 95% confidence intervals. The three colored thick bars are the group averages in both panels, with the shaded area showing the CI. We confirmed group differences using mixed-effects models that control for stimulus variance, participant variance, and also for participant's prior AI experience. Panel (B) displays participant-level data, variant means averaged for each participant; standard deviations of these averages by group are shown as black bars. There is more participant variance in the creative task, and more so among artists than laypeople. Panel (C) displays the entire dataset, arranged by participants (dot columns) according to their mean task performance (line of rectangles). GPT-4o does not produce the best works but does better on average than laypeople in the copying and both groups in the creative task.
• Figure 4: Examples of most successful trials across the experiment; cosine similarity in the corners. Panel (A): closest copies for each reference image (left, black border). (B): the most successful creative results, diverging furthest from the reference, either by managing to shift the style, hide the prefixed subject, or transform it. A larger version of this graph, comparing the best to the worst results, can be found in the Appendix.
• Figure 5: How do people write prompts? (A) Coefficients with 95% confidence intervals, from a linear regression model predicting the (z-scored cosine) success score: higher values indicate better result. Orange color indicates significant effects ($p<0.05$). All variables except length are categorical, with no presence as the reference level (presence predicts the x-axis value of increase in standard deviations in score). Prompt length in characters is scaled separately for the tasks; a $1\sigma$ increase in length predicts a $0.18\sigma$ increase in score. Panel (B) shows the relative usage frequency of the same components by the two groups, and (C) shows the sentiments towards AI reported by our experiment participants.