How to Prompt? Opportunities and Challenges of Zero- and Few-Shot Learning for Human-AI Interaction in Creative Applications of Generative Models

TL;DR

The paper argues that prompting—especially zero- and few-shot learning—offers a powerful, flexible paradigm for human-AI interaction in creative tasks but remains hard to control due to limited user guidance and opaque UI support. Through two HCI-driven brainstorming sessions, the authors identify opportunities (end-user tool creation, expanded expressiveness, and feedback/inspiration) and challenges (trial-and-error behavior, task representation, and computational/ethical concerns). They propose four design goals for prompting-enabled UIs and illustrate practical interfaces focusing on creative writing, including prompt formulation, combination, application, and representation. The work provides a structured starting point for researchers and designers to develop user-centered interfaces that democratize access to generative models and enhance collaborative creativity between humans and AI.

Abstract

Deep generative models have the potential to fundamentally change the way we create high-fidelity digital content but are often hard to control. Prompting a generative model is a promising recent development that in principle enables end-users to creatively leverage zero-shot and few-shot learning to assign new tasks to an AI ad-hoc, simply by writing them down. However, for the majority of end-users writing effective prompts is currently largely a trial and error process. To address this, we discuss the key opportunities and challenges for interactive creative applications that use prompting as a new paradigm for Human-AI interaction. Based on our analysis, we propose four design goals for user interfaces that support prompting. We illustrate these with concrete UI design sketches, focusing on the use case of creative writing. The research community in HCI and AI can take these as starting points to develop adequate user interfaces for models capable of zero- and few-shot learning.

Figures (6)

• Figure 1: In this GUI example, users enter a prompt in natural language and the system automatically parses this input. Key parameters such as detected task and task parameters can thus be edited directly. Then, the system automatically creates the (refined) text prompt for the generative model.
• Figure 2: In this example GUI, users create a prompt not by writing from scratch but by selecting from predefined "building blocks" that have been proven to work well and cover typical use case-specific aspects. Free entry is still supported as well.
• Figure 3: This example GUI focuses on prompt exploration and combination: Users write prompts to direct a "narrative tree" showing multiple possible responses to each prompt. Users select some of them as context for the next prompt(s), which direct the narrative further.
• Figure 4: This GUI example supports users in applying prompts by enabling them to write and save prompts as tools in a toolbar. Users can then (re-)apply the text transformations defined by these prompts to text selections, such as the "reformulate friendly" tool here.
• Figure 5: This example GUI supports asynchronous human-AI collaboration and in doing so also addresses potential delays of computationally costly prompts: The user can insert prompts into a text document anywhere, which trigger asynchronous calls to the LLM system, for example, to extend a part of the text. In the meantime, the user can continue working on another part of the text.