AIDED: Augmenting Interior Design with Human Experience Data for Designer-AI Co-Design

TL;DR

This paper tackles the challenge of encoding clients' lived experiences in interior design and how to integrate those signals into generative AI workflows without eroding professional autonomy. It introduces AIDED, a multimodal client-data framework that combines demographics, gaze traces, questionnaire responses, and AI-predicted overlays within a GAI editing loop, tested in a within-subject study with 12 professionals across four modalities. Key findings show questionnaire data are trusted and helpful for decision-making, gaze heatmaps impose cognitive load, and AI overlays require natural-language explanations to gain credibility, revealing an authenticity–interpretability trade-off. The study contributes a practical system, empirical insights into modality effects on co-design, and implications for AI tools that support human–data interaction in creative practice, informing future design of AI-mediated spatial-design workflows.

Abstract

Interior design often struggles to capture the subtleties of client experience, leaving gaps between what clients feel and what designers can act upon. We present AIDED, a designer-AI co-design workflow that integrates multimodal client data into generative AI (GAI) design processes. In a within-subjects study with twelve professional designers, we compared four modalities: baseline briefs, gaze heatmaps, questionnaire visualizations, and AI-predicted overlays. Results show that questionnaire data were trusted, creativity-enhancing, and satisfying; gaze heatmaps increased cognitive load; and AI-predicted overlays improved GAI communication but required natural language mediation to establish trust. Interviews confirmed that an authenticity-interpretability trade-off is central to balancing client voices with professional control. Our contributions are: (1) a system that incorporates experiential client signals into GAI design workflows; (2) empirical evidence of how different modalities affect design outcomes; and (3) implications for future AI tools that support human-data interaction in creative practice.

Figures (12)

• Figure 1: Overview of the five-phase study procedure. Phase 1 is a formative study with professional designers (Section \ref{['sec:formative']}). Phase 2 is the main scene-editing task with client data, including designer self-ratings, design outputs, and task evaluation (Sections \ref{['sec:system_workflow']}, \ref{['sec:task']}, \ref{['sec:task_evaluation']}, and \ref{['sec:DesignOutput_evaluation']}). Phase 3 introduces an exploratory prototype that provides LLM-based explanations for the AI-predicted heatmaps (Section \ref{['sec:LLM']}). Phase 4 covers the overall evaluation and post-task interviews with designers (Section \ref{['sec:overall_evaluation']}). Phase 5 reports an online survey and novice evaluation of the generated designs (Section \ref{['sec:Novice_evaluation']}).
• Figure 2: Client-information modalities across four experimental conditions. Matrix summarizing which types of client information are available in each condition. C1 (baseline, gray) provides only basic demographic data. C2 (purple) adds verbal feedback and real gaze heatmaps. C3 (orange) includes demographics, verbal feedback, and questionnaire-based visualizations. C4 (blue) integrates all modalities, adding AI-predicted attention maps on top of demographic data, verbal feedback, and questionnaire visualizations. Checkmarks indicate which data types are available.
• Figure 3: AIDED system interface for designer–AI co-design. (A) Main canvas showing the current interior design alongside navigation video frames spanning four scenes and a history of iterations, with heatmap overlays visible in Conditions 2 and 4. (B) Client information panel showing demographics (all conditions), verbal feedback (Conditions 2-4) and questionnaire responses (Conditions 3-4), with tabs for overall impressions and room-specific comments. (C) Control panel for AI-predicted architectural experience preference attention maps (Condition 4). (D) Prompt input box where designers describe how they want the AI to modify the scene; edited versions appear as new thumbnails for iterative refinement.
• Figure 4: LLM-generated interpretation of AI-predicted Overlays. When enabled, the LLM Interpretation panel (E) summarizes the current scene, explains the meaning of the AI-predicted attention overlay using the selected metric (architectural experience), and proposes concrete design suggestions. The left column provides a narrative scene overview; the middle column identifies the spatial regions highlighted in the heatmap; and the right column lists actionable design suggestions that designers can adapt for their prompts.
• Figure 5: Workflow of the AIDED generative AI design task. Step 1: Designers select one of four experimental conditions, each offering different levels of client information (C1–C4). Step 2: Designers select an interior style and a specific scene frame, and review the original design before any modification. Step 3: Designers view client information, including human experience data (demographics, gaze heatmaps, verbal feedback, and questionnaire visualizations, depending on the condition) and, in C4, AI-predicted attention maps. Step 4: Designers iteratively refine the interior by entering prompts; up to five edited versions can be generated, and any previous version can be used as the starting point for the next iteration. Step 5: Designers select a final design from all generated iterations.