ClearFairy: Capturing Creative Workflows through Decision Structuring, In-Situ Questioning, and Rationale Inference

TL;DR

ClearFairy is introduced, a think-aloud AI assistant for UI design that detects weak explanations, asks lightweight clarifying questions, and infers missing rationales, and structures reasoning into cognitive decision steps-linked units of actions, artifacts, and explanations making decisions traceable with generative AI.

Abstract

Capturing professionals' decision-making in creative workflows (e.g., UI/UX) is essential for reflection, collaboration, and knowledge sharing, yet existing methods often leave rationales incomplete and implicit decisions hidden. To address this, we present the CLEAR approach, which structures reasoning into cognitive decision steps-linked units of actions, artifacts, and explanations making decisions traceable with generative AI. Building on CLEAR, we introduce ClearFairy, a think-aloud AI assistant for UI design that detects weak explanations, asks lightweight clarifying questions, and infers missing rationales. In a study with twelve professionals, 85% of ClearFairy's inferred rationales were accepted (as-is or with revisions). Notably, the system increased "strong explanations"-rationales providing sufficient causal reasoning-from 14% to 83% without adding cognitive demand. Furthermore, exploratory applications demonstrate that captured steps can enhance generative AI agents in Figma, yielding predictions better aligned with professionals and producing coherent outcomes. We release a dataset of 417 decision steps to support future research.

Figures (12)

• Figure 1: Three main components (Explanation, Action, Artifact) for constructing Cognitive Decision Steps in Clear. Explanations provide the rationale behind several actions, and artifacts capture the resulting changes of each action.
• Figure 2: Construction of cognitively linked explanations. Cognitively linked explanations are created when consecutive utterances refer to the same goal or element within the workflow.
• Figure 3: Construction of cognitively linked actions with artifacts. Cognitively linked actions are formed when the same action is performed on multiple elements, or when a sequence of actions is performed on the same element.
• Figure 4: Formation of a Cognitive Decision Step by linking explanations with linked actions and artifacts. A cognitive link is established when an explanation directly refers to an associated action or artifact's part.
• Figure 5: ClearFairy as a Figma plugin. Users can capture their workflow by clicking the recording icon, as shown in (a) and (b). (c) shows the interface when a clarification question is generated by ClearFairy.