Artworks Reimagined: Exploring Human-AI Co-Creation through Body Prompting

TL;DR

Artworks Reimagined investigates body prompting as a pose-based input for generative AI to foster public engagement with artwork at events. The authors design a modular AI pipeline combining ControlNet pose control, style transfer, and CLIP-based prompt generation, deployed in a public event with a private booth option. They identify three embodied interaction strategies (imitate, reimagine, be expressive) and report high engagement, while noting safety, privacy, and expectancy considerations. The work provides design guidelines for public GLAM deployments and contributes to understanding human AI co-creation in situ.

Abstract

Image generation using generative artificial intelligence has become a popular activity. However, text-to-image generation - where images are produced from typed prompts - can be less engaging in public settings since the act of typing tends to limit interactive audience participation, thereby reducing its suitability for designing dynamic public installations. In this article, we explore body prompting as input modality for image generation in the context of installations at public event settings. Body prompting extends interaction with generative AI beyond textual inputs to reconnect the creative act of image generation with the physical act of creating artworks. We implement this concept in an interactive art installation, Artworks Reimagined, designed to transform existing artworks via body prompting. We deployed the installation at an event with hundreds of visitors in a public and private setting. Our semi-structured interviews with a sample of visitors (N = 79) show that body prompting was well-received and provides an engaging and fun experience to the installation's visitors. We present insights into participants' experience of body prompting and AI co-creation and identify three distinct strategies of embodied interaction focused on re-creating, reimagining, or casual interaction. We provide valuable recommendations for practitioners seeking to design interactive generative AI experiences in museums, galleries, and public event spaces.

Figures (5)

• Figure 1: Application architecture with inputs (left), textual prompts and body prompt (middle), and output (right).
• Figure 2: The study setup of the interactive art installation, Artworks Reimagined, guides participants through four application screens (top) in which participants give consent , select an artwork , body prompt via posing in front of a camera , and exit the camera booth . This is followed by an interview in which the results are revealed .
• Figure 3: The study setup consisted of four separate areas: 1) the public viewing and interview area, 2) the public posing stage with touch-screen display, 3) the private photo booth with touch-screen display, and 4) the private viewing area.
• Figure 4: Example of body prompts (middle row) and resulting images (bottom row) created during the event, together with the respective source artworks (top row). An online gallery of all images is available at https://www.artworksreimagined.com/.
• Figure 5: Examples of a change in narrative originating from the user (top row), the generative model (middle row), and both (bottom row), with the respective source artwork (top right) and body prompt (bottom right). Hallucinations from the model included, for instance, additional characters in the generated images (middle row, right, and bottom row, left), but also more subtle changes, such as a change in gender of a subject (middle row, center), face masks (bottom row, right), or a man walking an apple as if it were a dog on a leash (middle row, left).