Functional Flexibility in Generative AI Interfaces: Text Editing with LLMs through Conversations, Toolbars, and Prompts

TL;DR

This work explores interaction with Large Language Models over four years, before and after the rise of general-purpose LLMs, and considers how different UIs shape users' access to the functional space of generative AI models.

Abstract

Prompting-based user interfaces (UIs) shift the task of defining and accessing relevant functions from developers to users. However, how UIs shape this flexibility has not yet been investigated explicitly. We explored interaction with Large Language Models (LLMs) over four years, before and after the rise of general-purpose LLMs: (1) Our survey (N=121) elicited how users envision to delegate writing tasks to AI. This informed a conversational UI design. (2) A user study (N=10) revealed that people regressed to using short command-like prompts. (3) When providing these directly as shortcuts in a toolbar UI, in addition to prompting, users in our second study (N=12) dynamically switched between specified and flexible AI functions. We discuss functional flexibility as a new theoretical construct and thinking tool. Our work highlights the value of moving beyond conversational UIs, by considering how different UIs shape users' access to the functional space of generative AI models.

Figures (23)

• Figure 1: We investigated text editing with generative text models in a project that covered four years from 2020 to 2024 and included the moment when generative AI became widely popular. This research project builds on four parts as depicted in the figure: (1) a formative survey to explore people's text editor usage and to elicit how users would delegate tasks to AI (N=121), (2) a prototype integrating conversational AI plus a user study (N=10), (3) an alternative prototype offering AI tools, again tested in a user study (N=12), (4) a theoretical view on UIs and AI with a focus on functional flexibility. In summary, we contribute a set of elicited samples of user intents, findings from our studies about AI use in text editors, and a theoretical perspective on how UIs shape functional flexibility.
• Figure 2: Two screenshots we presented to survey participants to demonstrate how a text editor could look like that allows for delegating tasks to an AI that has the capability to carry them out. (a) Displays the AI appearing on the document like a user (top right icons), while (b) displays a UI element with a written task delegation to AI. We intentionally decoupled the visual representation of the task delegation from the editor to put focus on the process of delegating the task, not on a specific UI design.
• Figure 3: Responses from our survey questions on how frequently users work and collaborate on text documents. Weekly work on text documents was reported by 83.47% (61 daily, 40 weekly) of participants, rather on factual texts than on creative ones. Collaboration on text documents happens for 57.01% (11 daily, 29 weekly, 29 monthly) of participants at least monthly.
• Figure 4: Overview of participants' responses to the survey questions on how frequently they use (a) certain text editor software, (b) certain text editor software for collaborating on text documents, and (c) devices to work on text documents. Participants reported using Microsoft Office most frequently, followed by Google Docs, also for collaborating on text documents. This usage happens most frequently on laptops and desktop devices.
• Figure 5: Overview of how frequently users in our survey use tools to communicate about collaborating on text documents with others a) external from text editors b) within text documents. Communication outside of the document happens most frequently via email. Within documents, a third of participants reported to use interactive comments weekly to communicate with other people collaborating on the document.