Canvil: Designerly Adaptation for LLM-Powered User Experiences

TL;DR

The paper tackles how designers can meaningfully engage with large language models as a design material by introducing designerly adaptation, a translational process that two-way translates design requirements to LLM behavior and vice versa. It operationalizes this concept with Canvil, a Figma widget, and validates it through formative interviews (12 designers) and a design study (17 designers in 6 groups), showing that designers can surface LLM behavior through adaptation and co-evolve designs and model behavior. The work demonstrates Canvil’s potential to foster collaboration, knowledge sharing, and practical workflows for designer-driven AI design, while also outlining limitations and areas for tooling improvement. Collectively, the study advances a design-code-centric approach to human-centered AI, emphasizing materiality, sociomaterial practices, and collaborative processes as essential to responsible LLM-powered UX development.

Abstract

Advancements in large language models (LLMs) are sparking a proliferation of LLM-powered user experiences (UX). In product teams, designers often craft UX to meet user needs, but it is unclear how they engage with LLMs as a novel design material. Through a formative study with 12 designers, we find that designers seek a translational process that enables design requirements to shape and be shaped by LLM behavior, motivating a need for designerly adaptation to facilitate this translation. We then built Canvil, a Figma widget that operationalizes designerly adaptation. We used Canvil as a probe to study designerly adaptation in a group-based design study (6 groups, N=17), finding that designers constructively iterated on both adaptation approaches and interface designs to enhance end-user interaction with LLMs. Furthermore, designers identified promising collaborative workflows for designerly adaptation. Our work opens new avenues for processes and tools that foreground designers' human-centered expertise when developing LLM-powered applications.

Figures (4)

• Figure 1: An overview of the Canvil interface. A: A blank Canvil with its property menu invoked, with some additional copies styled with pre-set color options. The property menu has options for styling (1), model response generation (2), and model settings configuration (3). The Main Form(4) contains text input fields for structured authoring of model behavioral specifications. The Playground Area(5) allows users to quickly test inputs and view model outputs. B:Canvil supports collaboration by default. Here, we see Figma users collaboratively authoring a Canvil titled "Yoda impersonator." Like any other native Figma object, Canvil also supports stateful duplication.
• Figure 2: The "Generate" and "Settings" options in a Canvil's property menu can lead to three panels that appear alongside the Canvil itself. A:Playground Mode for response generation, where the input is read from and written to the Canvil's Playground Area. B:Design Mode for response generation, where the input is read from and written to design(s) on the Figma canvas. C: Settings for selecting and configuring the LLM.
• Figure 3: The setup for an individual participant's canvas in our study's Figma file. A: Informational packet containing descriptions of three user groups residing in North America (primarily west coast of the U.S.), the Middle East (primarily Turkey), and Asia (primarily India), respectively. B: Starter UIs for Feasto's 3-course meal planner with example user inputs to lower the barrier for testing model responses, along with basic UI elements such as text and buttons. C: Blank Canvils for participants to adapt LLMs.
• Figure 4: Examples of UI screens designers created for Feasto's 3-course meal planner during the study.