DesignWeaver: Dimensional Scaffolding for Text-to-Image Product Design

TL;DR

DesignWeaver introduces dimensional scaffolding to help novices craft more domain-specific prompts by surfacing key product-design dimensions from generated images into a manipulable palette. The system integrates GPT-4o for tag/prompt management with DALL-E 3 for image synthesis, enabling a bidirectional loop between visuals and language. An empirical comparison shows DesignWeaver yields longer, more nuanced prompts and greater design diversity, with expert evaluations rating higher novelty for final designs, though outputs can fall short of elevated user expectations. The work highlights implications for AI-assisted product design tools, including balancing guidance with creative freedom and aligning model capabilities with user aspirations. Overall, the approach demonstrates a path to bridge novice and expert design practices through visual-language scaffolding, while outlining opportunities for improved fidelity and collaborative workflows.

Abstract

Generative AI has enabled novice designers to quickly create professional-looking visual representations for product concepts. However, novices have limited domain knowledge that could constrain their ability to write prompts that effectively explore a product design space. To understand how experts explore and communicate about design spaces, we conducted a formative study with 12 experienced product designers and found that experts -- and their less-versed clients -- often use visual references to guide co-design discussions rather than written descriptions. These insights inspired DesignWeaver, an interface that helps novices generate prompts for a text-to-image model by surfacing key product design dimensions from generated images into a palette for quick selection. In a study with 52 novices, DesignWeaver enabled participants to craft longer prompts with more domain-specific vocabularies, resulting in more diverse, innovative product designs. However, the nuanced prompts heightened participants' expectations beyond what current text-to-image models could deliver. We discuss implications for AI-based product design support tools.

Figures (21)

• Figure 1: Overview of the iterative design process using DesignWeaver. The process involves four main stages: (1) Ingest the design document to extract initial dimensions and tags, (2) Refine and recommend dimensions to generate prompts, (3) Use prompts to render and refine images, and (4) Iterate based on new dimensions and tags inspired by the generated images.
• Figure 2: User Interface of DesignWeaver. The UI facilitates structured dimensional tagging and interactive exploration of AI-generated designs. Key features include a design document for guidance, a prompt box for input, a dimension palette for organizing and modifying design aspects, and an image panel displaying generated outputs. Users can add or delete dimensions, tag designs, view detailed image information, and curate favorite designs for final selection. This workflow supports iterative refinement and creativity.
• Figure 3: Number of user study participants with diverse experience in design, large language models, and text-to-image models.
• Figure 4: The baseline interface mimics a standard text-to-image setup, excluding scaffolding components.
• Figure 5: Workflow of the user study.