Understanding Nonlinear Collaboration between Human and AI Agents: A Co-design Framework for Creative Design

TL;DR

A subconscious change in people’s attitudes towards AI agents is noticed, shifting from perceiving them as mere executors to regarding them as opinionated colleagues, which effectively fostered the exploration and reflection processes of individual designers.

Abstract

Creative design is a nonlinear process where designers generate diverse ideas in the pursuit of an open-ended goal and converge towards consensus through iterative remixing. In contrast, AI-powered design tools often employ a linear sequence of incremental and precise instructions to approximate design objectives. Such operations violate customary creative design practices and thus hinder AI agents' ability to complete creative design tasks. To explore better human-AI co-design tools, we first summarize human designers' practices through a formative study with 12 design experts. Taking graphic design as a representative scenario, we formulate a nonlinear human-AI co-design framework and develop a proof-of-concept prototype, OptiMuse. We evaluate OptiMuse and validate the nonlinear framework through a comparative study. We notice a subconscious change in people's attitudes towards AI agents, shifting from perceiving them as mere executors to regarding them as opinionated colleagues. This shift effectively fostered the exploration and reflection processes of individual designers.

Figures (8)

• Figure 1: Creative design is a nonlinear process with an open-ended goal where almost no designer follows a predefined procedure. Designers search for multiple design options and jump from one sub-solution to another as they actively review, remix, and iterate them until achieving satisfactory results.
• Figure 2: In the traditional linear human-AI collaboration, AI agents execute a series of gradual and precise commands to achieve final design outcomes. We propose a nonlinear human-AI co-design framework characterized by facilitating a communication process before producing visual results and accommodating versatile actions for multiple choices.
• Figure 3: User interactions in OptiMuse follow these steps: (1) users review slides on the preview window; (2) users input commands and engage in rule-based conversation with OptiMuse; (3) OptiMuse generates alternatives as selective choices upon reaching convergence with users; (4) users remix choices through rule-based conversation; (5) users are satisfied with the choices and choose one option after several rounds of steps 1 to 4 (Notes: A1-10 refer to the AI agent actions defined in Sec. \ref{['sec:Ax']}).
• Figure 4: An overview of the scenario of Study II.
• Figure 5: Overview of the baseline in Study II, Copilot: (a) The preview window shows the current outcome; (b) Users can type commands to Copilot; (c) Copilot produces a single visual outcome in response to each command.