Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Expanding the Generative AI Design Space through Structured Prompting and Multimodal Interfaces

Nimisha Karnatak, Adrien Baranes, Rob Marchant, Huinan Zeng, Tríona Butler, Kristen Olson

TL;DR

Small business owners face difficulties translating brand intent into prompts and maintaining control over generated content. The authors propose ACAI, a multimodal, panel-based interface that composes a structured 'super prompt' from Branding, Audience & Goals, and Inspiration Board inputs, guided by Gemini 1.5 Pro, to produce brand-aligned ad briefs rather than raw visuals. A formative study with six SBOs identifies promptability, brand alignment, and user control as key pain points, which ACAI addresses through context-rich, constraint-aware design. The work demonstrates that structured, multimodal interaction can broaden accessibility and improve co-creative workflows in advertising, suggesting a shift from traditional prompt-based interfaces toward more inclusive AI design paradigms.

Abstract

Text-based prompting remains the predominant interaction paradigm in generative AI, yet it often introduces friction for novice users such as small business owners (SBOs), who struggle to articulate creative goals in domain-specific contexts like advertising. Through a formative study with six SBOs in the United Kingdom, we identify three key challenges: difficulties in expressing brand intuition through prompts, limited opportunities for fine-grained adjustment and refinement during and after content generation, and the frequent production of generic content that lacks brand specificity. In response, we present ACAI (AI Co-Creation for Advertising and Inspiration), a multimodal generative AI tool designed to support novice designers by moving beyond traditional prompt interfaces. ACAI features a structured input system composed of three panels: Branding, Audience and Goals, and the Inspiration Board. These inputs allow users to convey brand-relevant context and visual preferences. This work contributes to HCI research on generative systems by showing how structured interfaces can foreground user-defined context, improve alignment, and enhance co-creative control in novice creative workflows.

Expanding the Generative AI Design Space through Structured Prompting and Multimodal Interfaces

TL;DR

Small business owners face difficulties translating brand intent into prompts and maintaining control over generated content. The authors propose ACAI, a multimodal, panel-based interface that composes a structured 'super prompt' from Branding, Audience & Goals, and Inspiration Board inputs, guided by Gemini 1.5 Pro, to produce brand-aligned ad briefs rather than raw visuals. A formative study with six SBOs identifies promptability, brand alignment, and user control as key pain points, which ACAI addresses through context-rich, constraint-aware design. The work demonstrates that structured, multimodal interaction can broaden accessibility and improve co-creative workflows in advertising, suggesting a shift from traditional prompt-based interfaces toward more inclusive AI design paradigms.

Abstract

Text-based prompting remains the predominant interaction paradigm in generative AI, yet it often introduces friction for novice users such as small business owners (SBOs), who struggle to articulate creative goals in domain-specific contexts like advertising. Through a formative study with six SBOs in the United Kingdom, we identify three key challenges: difficulties in expressing brand intuition through prompts, limited opportunities for fine-grained adjustment and refinement during and after content generation, and the frequent production of generic content that lacks brand specificity. In response, we present ACAI (AI Co-Creation for Advertising and Inspiration), a multimodal generative AI tool designed to support novice designers by moving beyond traditional prompt interfaces. ACAI features a structured input system composed of three panels: Branding, Audience and Goals, and the Inspiration Board. These inputs allow users to convey brand-relevant context and visual preferences. This work contributes to HCI research on generative systems by showing how structured interfaces can foreground user-defined context, improve alignment, and enhance co-creative control in novice creative workflows.

Paper Structure

This paper contains 14 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Contextualization in Generative and Interactive Systems
  4. Prompting Challenges and Interface Design for Novice Users
  5. Formative Study
  6. Key Findings from Formative Research
  7. Design Requirements
  8. ACAI: AI Co-Creation for Advertising and Inspiration
  9. System Overview
  10. Implementation walkthrough
  11. Discussion
  12. Reimagining generative workflows
  13. Toward Inclusive AI Interface Design
  14. Conclusion and Future Work

Figures (3)

  • Figure 1: Formative Study: 6 UK SBOs identify visual elements they would like to incorporate into their branding.
  • Figure 2: ACAI Architecture
  • Figure 3: ACAI Interface featuring the three main panels: Branding (A-E), Audience & Goals (F-H), and Inspiration Board (I-L) that structure user input for generating brand-aligned advertisement briefs.