GenFaceUI: Meta-Design of Generative Personalized Facial Expression Interfaces for Intelligent Agents

TL;DR

This work addresses the design challenges of run-time generative facial expression interfaces for intelligent agents. It introduces the Generative Personalized Facial Expression Interface (GPFEI) framework and GenFaceUI, a meta-design tool that enables designers to craft templates, semantic tags, rules, and context mappings for personalized and context-aware expressions. A qualitative designer study reveals gains in controllability and consistency while highlighting needs for structured visual tooling, explanations, and clearer role allocation between designers and AI. Together, these contributions advance a meta-design perspective on generative interfaces and outline actionable directions for real-world deployment and future research.

Abstract

This work investigates generative facial expression interfaces for intelligent agents from a meta-design perspective. We propose the Generative Personalized Facial Expression Interface (GPFEI) framework, which organizes rule-bounded spaces, character identity, and context--expression mapping to address challenges of control, coherence, and alignment in run-time facial expression generation. To operationalize this framework, we developed GenFaceUI, a proof-of-concept tool that enables designers to create templates, apply semantic tags, define rules, and iteratively test outcomes. We evaluated the tool through a qualitative study with twelve designers. The results show perceived gains in controllability and consistency, while revealing needs for structured visual mechanisms and lightweight explanations. These findings provide a conceptual framework, a proof-of-concept tool, and empirical insights that highlight both opportunities and challenges for advancing generative facial expression interfaces within a broader meta-design paradigm.

Figures (10)

• Figure 1: Overview of the study: (A) the proposed GPFEI (Generative Personalized Facial Expression Interface) framework for intelligent agents; (B) GenFaceUI, a meta-design tool that operationalizes GPFEI; and (C) a qualitative study with designers using GenFaceUI across three design tasks.
• Figure 2: The GPFEI framework, highlighting personalized face customization and context-driven facial expressions under meta-design.
• Figure 3: GenFaceUI Architecture aligned with GPFEI's two meta-design capabilities: personalized face generation and contextual facial expression generation. The backend assembles prompts from four modules (Default Prompt, SVG Template, Design Rules, Context Rules), while the frontend provides Design Mode and Test Mode for iterative design--test--select.
• Figure 4: Interface overview with two modes: Design (1) and Test (2). In Design Mode, the Face Template Canvas (B1) and Toolbar (B2) compose the SVG template; the Semantic Tag Editor (C) applies tags and auto-adds default rules; the Design Rule Editor (D1) and Context Mapping Rule Editor (D2) define design and context-mapping rules. In Test Mode, designers provide context inputs (E1) and inspect generated outputs (E2) in the Runtime Expression Simulator (E), with optional device deploy (E3). The Phase Toggle (A) switches between personalized face generation and contextual facial expression generation.
• Figure 5: Face Template Canvas and Toolbar: (a) Selection Tool allows alignment, positioning, and scaling of elements; (b) Drawing Tools provide Area Drawing to define placement ranges and Shape Drawing to create geometric shapes.