Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Beyond Prompts: Exploring the Design Space of Mixed-Initiative Co-Creativity Systems

Zhiyu Lin, Upol Ehsan, Rohan Agarwal, Samihan Dani, Vidushi Vashishth, Mark Riedl

TL;DR

This paper defines a hypothetical human-AI configuration design space consisting of different means for humans and AI systems to communicate creative intent to each other, and finds that MI-CC systems with more extensive coverage of the design space are rated higher or on par on a variety of creative and goal-completion metrics.

Abstract

Generative Artificial Intelligence systems have been developed for image, code, story, and game generation with the goal of facilitating human creativity. Recent work on neural generative systems has emphasized one particular means of interacting with AI systems: the user provides a specification, usually in the form of prompts, and the AI system generates the content. However, there are other configurations of human and AI coordination, such as co-creativity (CC) in which both human and AI systems can contribute to content creation, and mixed-initiative (MI) in which both human and AI systems can initiate content changes. In this paper, we define a hypothetical human-AI configuration design space consisting of different means for humans and AI systems to communicate creative intent to each other. We conduct a human participant study with 185 participants to understand how users want to interact with differently configured MI-CC systems. We find out that MI-CC systems with more extensive coverage of the design space are rated higher or on par on a variety of creative and goal-completion metrics, demonstrating that wider coverage of the design space can improve user experience and achievement when using the system; Preference varies greatly between expertise groups, suggesting the development of adaptive, personalized MI-CC systems; Participants identified new design space dimensions including scrutability -- the ability to poke and prod at models -- and explainability.

Beyond Prompts: Exploring the Design Space of Mixed-Initiative Co-Creativity Systems

TL;DR

This paper defines a hypothetical human-AI configuration design space consisting of different means for humans and AI systems to communicate creative intent to each other, and finds that MI-CC systems with more extensive coverage of the design space are rated higher or on par on a variety of creative and goal-completion metrics.

Abstract

Generative Artificial Intelligence systems have been developed for image, code, story, and game generation with the goal of facilitating human creativity. Recent work on neural generative systems has emphasized one particular means of interacting with AI systems: the user provides a specification, usually in the form of prompts, and the AI system generates the content. However, there are other configurations of human and AI coordination, such as co-creativity (CC) in which both human and AI systems can contribute to content creation, and mixed-initiative (MI) in which both human and AI systems can initiate content changes. In this paper, we define a hypothetical human-AI configuration design space consisting of different means for humans and AI systems to communicate creative intent to each other. We conduct a human participant study with 185 participants to understand how users want to interact with differently configured MI-CC systems. We find out that MI-CC systems with more extensive coverage of the design space are rated higher or on par on a variety of creative and goal-completion metrics, demonstrating that wider coverage of the design space can improve user experience and achievement when using the system; Preference varies greatly between expertise groups, suggesting the development of adaptive, personalized MI-CC systems; Participants identified new design space dimensions including scrutability -- the ability to poke and prod at models -- and explainability.
Paper Structure (22 sections, 4 figures, 2 tables)

This paper contains 22 sections, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Design Space for MI-CC Systems
  4. Experimental System Overview
  5. Creative Context: Storytelling Domain and AI Algorithms
  6. Plug and Blend
  7. CARP
  8. Communications
  9. Elaboration Communication Modules
  10. Reflection Communcation Modules
  11. Miscellaneous Communication Modules
  12. Experience Manager and Frontend
  13. Study Methodology
  14. Results and Discussions
  15. Participant Creative Background
  16. ...and 7 more sections

Figures (4)

  • Figure 1: The design space of user-AI communications in mixed-initiative co-creative systems considered in this work, consisting of three dimensions. The blue cubes have been explored in prior studies lin_creative_2022.
  • Figure 2: The full system and six ablations---each removing one dimension from the design space---used as conditions in this work.
  • Figure 3: Screenshot of our experiment system, along with instructions.
  • Figure 4: Correlation between questions in the survey.