Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

UXCascade: Scalable Usability Testing with Simulated User Agents

Steffen Holter, Eunyee Koh, Mustafa Doga Dogan, Gromit Yeuk-Yin Chan

TL;DR

UXCascade addresses the bottlenecks of traditional usability testing by enabling scalable, on-demand evaluation using persona-driven AI agents. It introduces a five-stage workflow and an integrated agent framework (Simulation, Annotation, Refinement) plus a visual interface to link reasoning traces to UI elements and support lightweight design edits in a closed simulation loop. A within-subject user study with eight UX professionals demonstrates that UXCascade performs comparably to human-generated feedback while integrating smoothly into early-stage workflows, reducing cognitive load and enabling rapid iteration. The work provides a practical, low-cost complement to human studies, with potential to broaden exploration of design variants and accelerate UX refinements across diverse personas.

Abstract

Simulated user agents are increasingly used in usability testing to support fast, iterative UX workflows, as they generate rich data such as action logs and think-aloud reasoning, but the unstructured nature of this output often obscures actionable insights. We present UXCascade, an interactive tool for extracting, aggregating, and presenting agent-generated usability feedback at scale. Our core contribution is a multi-level analysis workflow that (1) highlights patterns across persona traits, goals, and outcomes, (2) links agent reasoning to specific issues, and (3) supports actionable design improvements. UXCascade operationalizes this approach by listing agent goals, traits, and issues in a structured overview. Practitioners can explore detailed reasoning traces and annotated views, propose interface edits, and assess their impact across personas. This enables a top-down, exploration-driven analysis from patterns to concrete UX interventions. A user study with eight UX professionals demonstrates that UXCascade integrates into existing workflows, enabling iterative feedback during early-stage interface development.

UXCascade: Scalable Usability Testing with Simulated User Agents

TL;DR

UXCascade addresses the bottlenecks of traditional usability testing by enabling scalable, on-demand evaluation using persona-driven AI agents. It introduces a five-stage workflow and an integrated agent framework (Simulation, Annotation, Refinement) plus a visual interface to link reasoning traces to UI elements and support lightweight design edits in a closed simulation loop. A within-subject user study with eight UX professionals demonstrates that UXCascade performs comparably to human-generated feedback while integrating smoothly into early-stage workflows, reducing cognitive load and enabling rapid iteration. The work provides a practical, low-cost complement to human studies, with potential to broaden exploration of design variants and accelerate UX refinements across diverse personas.

Abstract

Simulated user agents are increasingly used in usability testing to support fast, iterative UX workflows, as they generate rich data such as action logs and think-aloud reasoning, but the unstructured nature of this output often obscures actionable insights. We present UXCascade, an interactive tool for extracting, aggregating, and presenting agent-generated usability feedback at scale. Our core contribution is a multi-level analysis workflow that (1) highlights patterns across persona traits, goals, and outcomes, (2) links agent reasoning to specific issues, and (3) supports actionable design improvements. UXCascade operationalizes this approach by listing agent goals, traits, and issues in a structured overview. Practitioners can explore detailed reasoning traces and annotated views, propose interface edits, and assess their impact across personas. This enables a top-down, exploration-driven analysis from patterns to concrete UX interventions. A user study with eight UX professionals demonstrates that UXCascade integrates into existing workflows, enabling iterative feedback during early-stage interface development.
Paper Structure (35 sections, 7 figures, 3 tables)

This paper contains 35 sections, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Usability Testing
  4. Simulated Agents
  5. Persona-Guided Simulation
  6. Requirements Analysis
  7. Formative Study
  8. Understanding UX Workflows
  9. User Needs and Challenges
  10. Description of Requirements
  11. Workflow
  12. UXCascade
  13. Agent Framework
  14. Visual Interface
  15. User Study
  16. ...and 20 more sections

Figures (7)

  • Figure 1: Overview of key stages in the UX development cycle and commonly used evaluation methods.
  • Figure 2: Overview of our workflow combining goal-driven exploration with iterative refinement. Users begin by examining agent goals and outcomes, analyzing behavior across persona traits, and identifying specific UX issues. They can then propose interface changes and assess their effects across diverse agent profiles.
  • Figure 3: Our agent framework comprising (1) Simulation Agents for generating persona-grounded interaction traces, (2) Annotation Agents for tagging cognitive intent and detecting usability issues, and (3) Refinement Agents for applying and evaluating targeted interface edits.
  • Figure 4: User interaction scenario with the UXCascade visual interface, showing (A) isolation of issue details, (B) generation of a fix based on natural language instruction, (C) simulation of new agent behavior, and (D) visualization of impacted personas across the user population.
  • Figure 5: Average number of usability issues identified by participants across conditions
  • ...and 2 more figures