The Way We Notice, That's What Really Matters: Instantiating UI Components with Distinguishing Variations

TL;DR

This work instantiates distinguishing variations in Celestial, a tool that helps developers explore and visualize distinguishing variations that are both mimetic and distinct, and frames distinguishing variation generation as design-space sampling.

Abstract

Front-end developers author UI components to be broadly reusable by parameterizing visual and behavioral properties. While flexible, this makes instantiation harder, as developers must reason about numerous property values and interactions. In practice, they must explore the component's large design space and provide realistic and natural values to properties. To address this, we introduce distinguishing variations: variations that are both mimetic and distinct. We frame distinguishing variation generation as design-space sampling, combining symbolic inference to identify visually important properties with an LLM-driven mimetic sampler to produce realistic instantiations from its world knowledge. We instantiate distinguishing variations in Celestial, a tool that helps developers explore and visualize distinguishing variations. In a study with front-end developers (n=12), participants found these variations useful for comparing and mapping component design spaces, reported that mimetic instantiations were domain-relevant, and validated that Celestial transformed component instantiation from a manual process into a structured, exploratory activity.

Figures (7)

• Figure 1: Four different valid instantiations of NotificationCenter component. Although the code to instantiate the variations only differ slightly from each other, visually and look very similar, while and look substantially different. While is a valid instantiation, it looks unnatural, violating real-world conventions.
• Figure 2: Variations grounded on sampling instructions. Celestial instantiates variations for a selectable and searchable notification center for a CRM app. Chloe notices that all the variations contain notification text based on CRM-specific data, which is searchable and selectable.
• Figure 3: Chloe uses Celestial to instantiate more examples to answer two questions: "Does it support right-to-left languages?" and "Does it support rendering media?"
• Figure 4: Celestial system architecture. The ComponentAnalyzer uses symbolic techniques to retrieve a list of highly visually-impactful properties of the UI component. The CoverageAnalyzer measures the coverage of the generated variations to inform the sampler. Finally, the MimeticSampler, powered by an LLM, uses the component analysis and the coverage metrics along with an optional sampling instruction from the developer to generate distinguishing variations.
• Figure 5: Celestial's inspect UI. The developer can choose to inspect a single variant in detail using Celestial's inspect feature. The center pane renders the selected variant. The right pane shows the property values that instantiate the selected variant. The user can also choose to inspect the Storybook.js code for the variant, and optionally edit the properties to immediately see the changes rendered in the center pane.