Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards A Diffractive Analysis of Prompt-Based Generative AI

Nina Rajcic, Maria Teresa Llano, Jon McCormack

TL;DR

This study investigates how prompt-based generative AI interfaces influence artists who work with physical materials by applying a diffractive methodology to seven artists using personalized Stable Diffusion models over two weeks. It identifies two dominant modes—AI for ideation and AI for production—and analyzes the implications for creative agency, authorship, efficiency, and materiality. The findings highlight ambivalence toward data provenance, ownership, and economic impact, and offer design guidelines that tailor AI collaboration to each mode while stressing the need for domain-specific CST integration. Overall, the work positions AI as a creative assistant that extends human capabilities, rather than replacing them, and calls for critical, ethically informed design as these tools become more pervasive in artistic practice.

Abstract

Recent developments in prompt-based generative AI has given rise to discourse surrounding the perceived ethical concerns, economic implications, and consequences for the future of cultural production. As generative imagery becomes pervasive in mainstream society, dominated primarily by emerging industry leaders, we encourage that the role of the CHI community be one of inquiry; to investigate the numerous ways in which generative AI has the potential to, and already is, augmenting human creativity. In this paper, we conducted a diffractive analysis exploring the potential role of prompt-based interfaces in artists' creative practice. Over a two week period, seven visual artists were given access to a personalised instance of Stable Diffusion, fine-tuned on a dataset of their work. In the following diffractive analysis, we identified two dominant modes adopted by participants, AI for ideation, and AI for production. We furthermore present a number of ethical design considerations for the future development of generative AI interfaces.

Towards A Diffractive Analysis of Prompt-Based Generative AI

TL;DR

This study investigates how prompt-based generative AI interfaces influence artists who work with physical materials by applying a diffractive methodology to seven artists using personalized Stable Diffusion models over two weeks. It identifies two dominant modes—AI for ideation and AI for production—and analyzes the implications for creative agency, authorship, efficiency, and materiality. The findings highlight ambivalence toward data provenance, ownership, and economic impact, and offer design guidelines that tailor AI collaboration to each mode while stressing the need for domain-specific CST integration. Overall, the work positions AI as a creative assistant that extends human capabilities, rather than replacing them, and calls for critical, ethically informed design as these tools become more pervasive in artistic practice.

Abstract

Recent developments in prompt-based generative AI has given rise to discourse surrounding the perceived ethical concerns, economic implications, and consequences for the future of cultural production. As generative imagery becomes pervasive in mainstream society, dominated primarily by emerging industry leaders, we encourage that the role of the CHI community be one of inquiry; to investigate the numerous ways in which generative AI has the potential to, and already is, augmenting human creativity. In this paper, we conducted a diffractive analysis exploring the potential role of prompt-based interfaces in artists' creative practice. Over a two week period, seven visual artists were given access to a personalised instance of Stable Diffusion, fine-tuned on a dataset of their work. In the following diffractive analysis, we identified two dominant modes adopted by participants, AI for ideation, and AI for production. We furthermore present a number of ethical design considerations for the future development of generative AI interfaces.
Paper Structure (21 sections, 5 figures, 1 table)

This paper contains 21 sections, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. AI and Creativity
  4. Co-Creative AI
  5. Diffractive Methodology
  6. Evaluation
  7. Personalising Stable Diffusion
  8. Study Design
  9. Diffractive Analysis
  10. Creative Agency
  11. Ideation
  12. Plagiarism
  13. Efficiency
  14. Identity and Authorship
  15. Materiality
  16. ...and 6 more sections

Figures (5)

  • Figure 1: An image used in finetuning, with human-labelled caption below (left), beside two generated images from the same model with prompts (right)
  • Figure 2: A screenshot of the web-app interface used for the personalised diffusion model
  • Figure 3: A selection of early attempts to generate a metallic flower (left), final attempts to generate a metallic flower (right)
  • Figure 4: A selection of AI generated images (top) recreated by hand (bottom) as a part Georgia's creative experimentation.
  • Figure 5: Generative images with prompts displayed underneath; an example of non-literal prompting