AlignUI: A Method for Designing LLM-Generated UIs Aligned with User Preferences

TL;DR

AlignUI addresses the challenge of aligning LLM-generated UIs with real user tasks and preferences. It introduces a lightweight inference-time method that injects a crowdsourced user-preference dataset into the LLM's reasoning and code-generation process. A 720-item dataset from 50 general users supports multi-criteria preferences (predictability, efficiency, explorability) and is used to generate UIs for image editing tasks; a user study with 78 participants on six unseen tasks shows improved alignment compared to LLMs without such guidance. The work demonstrates generalizability to related tasks and outlines directions for broader domains and personalized UI design.

Abstract

Designing user interfaces that align with user preferences is a time-consuming process, which requires iterative cycles of prototyping, user testing, and refinement. Recent advancements in LLM-based UI generation have enabled efficient UI generation to assist the UI design process. We introduce AlignUI, a method that aligns LLM-generated UIs with user tasks and preferences by using a user preference dataset to guide the LLM's reasoning process. The dataset was crowdsourced from 50 general users (the target users of generated UIs) and contained 720 UI control preferences on eight image-editing tasks. We evaluated AlignUI by generating UIs for six unseen tasks and conducting a user study with 72 additional general users. The results showed that the generated UIs closely align with multiple dimensions of user preferences. We conclude by discussing the applicability of our method to support user-aligned UI design for multiple task domains and user groups, as well as personalized user needs.

Figures (16)

• Figure 1: Example UI in our study (formative and crowdsourcing) for adjusting image hue to fall colors. It presents task description and requirements, multiple UI controls to perform the task, and a user preference selection panel for multiple user preference aspects (predictability, efficiency, explorability).
• Figure 2: AlignUI method pipeline. The pipeline takes user context as input, which specifies the user's task description and UI control preferences. With this input, the LLM is additionally fed with a user preference dataset, which contains a set of tasks and user-preferred UI controls to perform these tasks regarding multiple user preference aspects, and UI control candidates, which are used to support UI control implementation. The LLM is prompted by multi-stage reasoning to obtain user preference-aligned UI controls. These reasoned UI controls are then implemented based on LLM-based code generation and presented to the user in a functional UI.
• Figure 3: Crowdsourcing results 1. User preference distribution of tasks 1-4 (task set 1).
• Figure 4: Crowdsourcing results 2. User preference distribution of tasks 5-8 (task set 2).
• Figure 5: Example of crowdsourced user preference data for task image_adjust_fall_color. The data is organized in the JSON format in our user preference dataset.