I Can Embrace and Avoid Vagueness Myself: Supporting the Design Process by Balancing Vagueness through Text-to-Image Generative AI

TL;DR

CLAY is an interactive system that balances vagueness through iterative prompt refinement by integrating the strengths of text-to-image generative AI.

Abstract

This study examines the role of vagueness in the design process and its strategic management for the effective human-AI interaction. While vagueness in the generation of design ideas promotes diverse interpretations and prevents fixation, excessive vagueness can lead to scattered results. Designers attempt to use image search tools or generative AIs (e.g., Dall-E) for their work but often fail to achieve satisfactory results because the level of vagueness is not properly managed in these technologies. In this work, we identified how designers coordinate vagueness in their design process and applied key components of the process to the design of CLAY, an interactive system that balances vagueness through iterative prompt refinement by integrating the strengths of text-to-image generative AI. Results from our user study with 10 fashion designers showed that CLAY effectively supported their design process, reducing design time, and expanding creative possibilities compared to their existing practice, by allowing them to both embrace and avoid vagueness as needed. Our study highlights the importance of identifying key characteristics of the target user and domain, and exploring ways to incorporate them into the design of an AI-based interactive tool.

Figures (12)

• Figure 1: The diagram shows the hierarchical and combinational structures used in the vagueness balancing process for both the moodboard and design stages. For each stage, the process begins with a hierarchical structure, and as users progress through the process, these elements are combined. The number of circles is a placeholder and the variety could be more extensive.
• Figure 2: (Left) A moodboard that is typically created by the designer. (Right) A sketch of the moodboard, which is likely to be expanded or modified (with different elements or structures) over time.
• Figure 3: The left image shows a sketch of how designs are derived from the moodboard. Subcomponents in the moodboard are likely to be modified depending on the level of vagueness. The right image shows a sample design based on the conventional method.
• Figure 4: Failure cases for balancing vagueness in both an image search tool (top) and a text-to-image generative AI (bottom). In both scenarios, numerous queries with no chance of getting to the next step lead to high user dissatisfaction and abandonment.
• Figure 5: The main process structure of CLAY consists of four stages: starting from the design concept, users enter a vague query that leads to hierarchical results. They then refine the prompt and iterate a few times until they reach the desired combination of results, achieving user satisfaction.