Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Feedforward in Generative AI: Opportunities for a Design Space

Bryan Min, Haijun Xia

TL;DR

The paper addresses the problem that users struggle to anticipate GenAI outputs, incurring high cognitive load due to feedback-centric interactions. It introduces the concept of feedforward in GenAI and demonstrates it through four instantiations across conversational UIs, document editors, malleable interfaces, and agent automations. Key contributions include identifying design dimensions (representation, detail level, manipulability) and proposing interactive feedforward mechanisms (outlines, minimaps, operation lists) to enable anticipation and prompt disambiguation. The work offers a framework for systematic design of GenAI interfaces with potential to reduce turn-taking, improve user meta-cognition, and broaden the applicability of GenAI across contexts.

Abstract

Generative AI (GenAI) models have become more capable than ever at augmenting productivity and cognition across diverse contexts. However, a fundamental challenge remains as users struggle to anticipate what AI will generate. As a result, they must engage in excessive turn-taking with the AI's feedback to clarify their intent, leading to significant cognitive load and time investment. Our goal is to advance the perspective that in order for users to seamlessly leverage the full potential of GenAI systems across various contexts, we must design GenAI systems that not only provide informative feedback but also informative feedforward -- designs that tell users what AI will generate before the user submits their prompt. To spark discussion on feedforward in GenAI, we designed diverse instantiations of feedforward across four GenAI applications: conversational UIs, document editors, malleable interfaces, and automation agents, and discussed how these designs can contribute to a more rigorous investigation of a design space and a set of guidelines for feedforward in all GenAI systems.

Feedforward in Generative AI: Opportunities for a Design Space

TL;DR

The paper addresses the problem that users struggle to anticipate GenAI outputs, incurring high cognitive load due to feedback-centric interactions. It introduces the concept of feedforward in GenAI and demonstrates it through four instantiations across conversational UIs, document editors, malleable interfaces, and agent automations. Key contributions include identifying design dimensions (representation, detail level, manipulability) and proposing interactive feedforward mechanisms (outlines, minimaps, operation lists) to enable anticipation and prompt disambiguation. The work offers a framework for systematic design of GenAI interfaces with potential to reduce turn-taking, improve user meta-cognition, and broaden the applicability of GenAI across contexts.

Abstract

Generative AI (GenAI) models have become more capable than ever at augmenting productivity and cognition across diverse contexts. However, a fundamental challenge remains as users struggle to anticipate what AI will generate. As a result, they must engage in excessive turn-taking with the AI's feedback to clarify their intent, leading to significant cognitive load and time investment. Our goal is to advance the perspective that in order for users to seamlessly leverage the full potential of GenAI systems across various contexts, we must design GenAI systems that not only provide informative feedback but also informative feedforward -- designs that tell users what AI will generate before the user submits their prompt. To spark discussion on feedforward in GenAI, we designed diverse instantiations of feedforward across four GenAI applications: conversational UIs, document editors, malleable interfaces, and automation agents, and discussed how these designs can contribute to a more rigorous investigation of a design space and a set of guidelines for feedforward in all GenAI systems.

Paper Structure

This paper contains 10 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Feedforward in Generative AI
  3. Feedforward in Conversational LLM UIs
  4. Feedforward Components
  5. Manipulating Feedforward Content
  6. Application Scenarios
  7. Document Editors
  8. Malleable Interfaces
  9. Agent Automations
  10. Future Work

Figures (3)

  • Figure 1: (1) When the user stops typing for a brief moment, the conversational UI presents two feedforward components: (a) a list of key topics and an outline, and (b) a visual minimap of the anticipated length of the response. (2) The user anticipates what the LLM will generate and adjusts their prompt to match it. (3) They also ask for less information, in which the feedforward components update to match the request.
  • Figure 2: Users can directly interact and manipulate feedforward components by for instance (A) deleting unneeded paragraphs, (B) expanding details about a section, and (C) transforming the representation of the expected content.
  • Figure 3: We explored three different applications for designing feedforward in GenAI systems: (A) document editors, (B) malleable interfaces, and (C) agent automation systems.