EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experience

TL;DR

EvoCUA tackles the data bottleneck in native computer-use agents by replacing static imitation with an evolving paradigm that blends verifiable synthesis, massive asynchronous sandbox rollout, and experience-driven policy optimization. A Verifiable Synthesis Engine generates solvable tasks with executable validators, while a Scalable Interaction Infrastructure enables tens of thousands of concurrent rollouts to feed an on-policy learning loop. The approach employs rejection sampling and step-level direct preference optimization to convert synthetic trajectories into robust policies, achieving state-of-the-art results on OSWorld among open-weight models. The work demonstrates scalable, generalizable advances for native GUI agents and outlines online RL as a promising path to closing remaining gaps toward fully autonomous computer use capabilities.

Abstract

The development of native computer-use agents (CUA) represents a significant leap in multimodal AI. However, their potential is currently bottlenecked by the constraints of static data scaling. Existing paradigms relying primarily on passive imitation of static datasets struggle to capture the intricate causal dynamics inherent in long-horizon computer tasks. In this work, we introduce EvoCUA, a native computer use agentic model. Unlike static imitation, EvoCUA integrates data generation and policy optimization into a self-sustaining evolutionary cycle. To mitigate data scarcity, we develop a verifiable synthesis engine that autonomously generates diverse tasks coupled with executable validators. To enable large-scale experience acquisition, we design a scalable infrastructure orchestrating tens of thousands of asynchronous sandbox rollouts. Building on these massive trajectories, we propose an iterative evolving learning strategy to efficiently internalize this experience. This mechanism dynamically regulates policy updates by identifying capability boundaries -- reinforcing successful routines while transforming failure trajectories into rich supervision through error analysis and self-correction. Empirical evaluations on the OSWorld benchmark demonstrate that EvoCUA achieves a success rate of 56.7%, establishing a new open-source state-of-the-art. Notably, EvoCUA significantly outperforms the previous best open-source model, OpenCUA-72B (45.0%), and surpasses leading closed-weights models such as UI-TARS-2 (53.1%). Crucially, our results underscore the generalizability of this approach: the evolving paradigm driven by learning from experience yields consistent performance gains across foundation models of varying scales, establishing a robust and scalable path for advancing native agent capabilities.

Figures (9)

• Figure 1: Performance comparison on the OSWorld-Verified benchmark. Our EvoCUA-32B achieves state-of-the-art performance (56.7%) among open-weights models.
• Figure 2: Overview of EvoCUA. The diagram illustrates the paradigm shift from static imitation to an active evolving experience learning cycle (center). The approach unifies three core modules: the Verifiable Synthesis Engine (top left); the Scalable Interaction Infrastructure (right); and Iterative Optimization (bottom left).
• Figure 3: Architecture of the Verifiable Synthesis Engine. The pipeline operates in three cascading stages: (1) Structured Task Space Construction to define diverse scenarios from domain taxonomies and hybrid resources; (2) Agentic Dual-Stream Synthesis, where a Task Architect (VLM) co-generates instructions ($g$) and executable validators ($V_g$) via a closed-loop feedback mechanism; and (3) Rigorous Quality Assurance to filter outputs for high consistency and ensure decontamination, yielding the final verifiable dataset.
• Figure 4: Scalable Infrastructure. The architecture orchestrates massive interaction requests from the online RL loop (top-left) through an asynchronous gateway and distributed scheduler (top-right). The bottom layer deploys parallel sandbox clusters, highlighting the Computer Use Sandbox, which utilizes QEMU-KVM virtualization and a calibrated OS to ensure input determinism, rendering consistency, and runtime stability for high-fidelity environments.
• Figure 5: Overview of the Dual-Paradigm DPO. The process begins at a critical forking point $t^*$. Paradigm I (Action Correction) establishes a preference for the chosen action $(z_w, a_w)$ over the rejected action $(z_l, a_l)$. Paradigm II (Reflection) addresses the deviated state at $t^*+1$, prioritizing Reflection over Blind Continuation. Both paradigms define preference pairs that optimize the DPO Loss $\mathcal{J}(\theta)$ to maximize the margin between effective and ineffective strategies.