Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Design Generative AI for Practitioners: Exploring Interaction Approaches Aligned with Creative Practice

Xiaohan Peng, Wendy E. Mackay, Janin Koch

TL;DR

It is argued that alignment is a dynamic negotiation, with AI adopting proactive or reactive roles according to designers' instrumental and inspirational needs and the creative stage.

Abstract

Design is a non-linear, reflective process in which practitioners engage with visual, semantic, and other expressive materials to explore, iterate, and refine ideas. As Generative AI (GenAI) becomes integrated into professional design practice, traditional interaction approaches focusing on prompts or whole-image manipulation can misalign AI output with designers' intent, forcing visual thinkers into verbal reasoning or post-hoc adjustments. We present three interaction approaches from DesignPrompt, FusAIn, and DesignTrace that distribute control across intent, input, and process, enabling designers to guide AI alignment at different stages of interaction. We further argue that alignment is a dynamic negotiation, with AI adopting proactive or reactive roles according to designers' instrumental and inspirational needs and the creative stage.

Design Generative AI for Practitioners: Exploring Interaction Approaches Aligned with Creative Practice

TL;DR

It is argued that alignment is a dynamic negotiation, with AI adopting proactive or reactive roles according to designers' instrumental and inspirational needs and the creative stage.

Abstract

Design is a non-linear, reflective process in which practitioners engage with visual, semantic, and other expressive materials to explore, iterate, and refine ideas. As Generative AI (GenAI) becomes integrated into professional design practice, traditional interaction approaches focusing on prompts or whole-image manipulation can misalign AI output with designers' intent, forcing visual thinkers into verbal reasoning or post-hoc adjustments. We present three interaction approaches from DesignPrompt, FusAIn, and DesignTrace that distribute control across intent, input, and process, enabling designers to guide AI alignment at different stages of interaction. We further argue that alignment is a dynamic negotiation, with AI adopting proactive or reactive roles according to designers' instrumental and inspirational needs and the creative stage.
Paper Structure (10 sections, 3 figures)

This paper contains 10 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Interaction Approaches for Creative Practice
  3. Intent Alignment: Configurable Multimodal Scaffolds
  4. Input Alignment: Direct Manipulable Visual Prompts
  5. Process Alignment: Iterative and Trackable Generative Workflows
  6. Rethinking Roles of AI: Alignment as a Shifting Partnership
  7. Proactive AI: Designing with Productive Friction
  8. Reactive AI: Designing with Instrumental Intent
  9. Shifting Initiative in Design Practice
  10. Conclusion

Figures (3)

  • Figure 1: DesignPrompt supports intent decomposition through multiple input modalities: (A) image inpainting, (B) color, and (C) semantic tags. Multimodal inputs are mapped to editable text prompts, enabling interactive prompt editing and reordering to refine textual input to GenAI.
  • Figure 2: FusAIn reifies image-based visual prompts as personalized “pens” that encode specific visual properties. With “texture pens” users extract a texture from a source image, load it into a pen, and draw to apply that texture onto target visuals. The system then generates an image that preserves other properties while consistently applying the texture.
  • Figure 3: DesignTrace: a GenAI design tool for exploring alternatives and tracking design progress. Designers can extract semantic attributes from inspirational images and system suggestions, apply them to selected regions or entire images, branch multiple alternatives under shared semantic categories, customize prompts for image generation, make localized edits while navigating editing history, and collapse unused versions.