UI-CUBE: Enterprise-Grade Computer Use Agent Benchmarking Beyond Task Accuracy to Operational Reliability

TL;DR

UI-CUBE addresses the gap between task-oriented CUA benchmarks and the reliability demanded by enterprise deployment by combining systematic UI-interaction coverage with realistic enterprise workflow mocks. It reveals a sharp capability cliff: agents perform well on simple interactions but fail to coordinate multi-step workflows, even with human baselines indicating practical ceilings. The benchmark uses execution-based, state-diff validation within a containerized, multi-resolution environment to ensure deterministic evaluation across resolutions and interfaces. These findings highlight fundamental architectural bottlenecks in memory management and planning, guiding future work toward production-ready CUAs capable of sustained enterprise process automation.

Abstract

While current Computer Use Agent (CUA) benchmarks measure task completion effectively, they provide limited assessment of enterprise deployment readiness, emphasizing functional correctness over the operational reliability required for production systems. We present UI-CUBE (UiPath Computer Use BEnchmark), a systematic benchmark comprising 226 tasks across two difficulty tiers designed to expose fundamental architectural limitations in current CUAs. Our evaluation covers simple UI interactions (136 tasks) and complex workflows including copy-paste tasks (50 tasks) and enterprise application scenarios (40 tasks), with systematic interface variation coverage, multi-resolution testing and automated validation of task success through the application state. Evaluation of five state-of-the-art models reveals a sharp capability cliff rather than gradual performance degradation. Simple UI interactions achieve 67-85% success rates (compared to 97.9% human performance), but complex workflows drop precipitously to 9-19%. Human evaluators with no prior application experience achieve only 61.2% on complex tasks despite near-perfect performance on simple tasks, establishing realistic performance ceilings. This discontinuous performance pattern -- where agents achieve 68-87% of human performance on simple tasks but only 15-32% on complex workflows -- indicates fundamental architectural limitations in memory management, hierarchical planning, and state coordination rather than incremental capability gaps addressable through better training or prompting. UI-CUBE functions as an enterprise-readiness diagnostic, revealing that while current CUAs can manipulate individual interface elements, they cannot yet function as reliable workflow automation tools. These findings provide architectural insights essential for developing production-ready CUAs capable of managing complex, multi-step enterprise processes.

Figures (8)

• Figure 1: Control distribution across simple scenarios showing systematic coverage of interactive elements.
• Figure 2: Structure distribution showing layout complexity with emphasis on forms, tables, and custom designs.
• Figure 3: Action distribution reflecting real-world automation patterns with selection operations dominating enterprise workflows.
• Figure 4: Agent miss-clicks the expander button, demonstrating grounding precision failures in hierarchical navigation contexts.
• Figure 5: Agent incorrectly reads text from a list, showing perception errors in structured data environments.