Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLMs as Scalable, General-Purpose Simulators For Evolving Digital Agent Training

Yiming Wang, Da Yin, Yuedong Cui, Ruichen Zheng, Zhiqian Li, Zongyu Lin, Di Wu, Xueqing Wu, Chenchen Ye, Yu Zhou, Kai-Wei Chang

TL;DR

UI-Simulator introduces an LLM-powered digital world model to synthesize diverse UI trajectories at scale, addressing data scarcity in training digital agents. It combines a multi-step state prediction, natural-language drafting, and structured state grounding, with retrieval-augmented variants and a stepwise guided rollout to ensure coherent, high-quality data. UI-Simulator-Grow adds targeted scaling by selecting high-impact tasks via teacher-forcing loss signals and generating diverse trajectory variants, enabled by continual learning, to achieve data-efficient gains. On WebArena and AndroidWorld, UI-Simulator matches or surpasses baselines using weaker teachers, with UI-Simulator-Grow achieving rapid improvements and approaching 70B-scale performance using only 66% of trajectories, highlighting strong scalability and robustness of the approach.

Abstract

Digital agents require diverse, large-scale UI trajectories to generalize across real-world tasks, yet collecting such data is prohibitively expensive in both human annotation, infra and engineering perspectives. To this end, we introduce $\textbf{UI-Simulator}$, a scalable paradigm that generates structured UI states and transitions to synthesize training trajectories at scale. Our paradigm integrates a digital world simulator for diverse UI states, a guided rollout process for coherent exploration, and a trajectory wrapper that produces high-quality and diverse trajectories for agent training. We further propose $\textbf{UI-Simulator-Grow}$, a targeted scaling strategy that enables more rapid and data-efficient scaling by prioritizing high-impact tasks and synthesizes informative trajectory variants. Experiments on WebArena and AndroidWorld show that UI-Simulator rivals or surpasses open-source agents trained on real UIs with significantly better robustness, despite using weaker teacher models. Moreover, UI-Simulator-Grow matches the performance of Llama-3-70B-Instruct using only Llama-3-8B-Instruct as the base model, highlighting the potential of targeted synthesis scaling paradigm to continuously and efficiently enhance the digital agents.

LLMs as Scalable, General-Purpose Simulators For Evolving Digital Agent Training

TL;DR

UI-Simulator introduces an LLM-powered digital world model to synthesize diverse UI trajectories at scale, addressing data scarcity in training digital agents. It combines a multi-step state prediction, natural-language drafting, and structured state grounding, with retrieval-augmented variants and a stepwise guided rollout to ensure coherent, high-quality data. UI-Simulator-Grow adds targeted scaling by selecting high-impact tasks via teacher-forcing loss signals and generating diverse trajectory variants, enabled by continual learning, to achieve data-efficient gains. On WebArena and AndroidWorld, UI-Simulator matches or surpasses baselines using weaker teachers, with UI-Simulator-Grow achieving rapid improvements and approaching 70B-scale performance using only 66% of trajectories, highlighting strong scalability and robustness of the approach.

Abstract

Digital agents require diverse, large-scale UI trajectories to generalize across real-world tasks, yet collecting such data is prohibitively expensive in both human annotation, infra and engineering perspectives. To this end, we introduce , a scalable paradigm that generates structured UI states and transitions to synthesize training trajectories at scale. Our paradigm integrates a digital world simulator for diverse UI states, a guided rollout process for coherent exploration, and a trajectory wrapper that produces high-quality and diverse trajectories for agent training. We further propose , a targeted scaling strategy that enables more rapid and data-efficient scaling by prioritizing high-impact tasks and synthesizes informative trajectory variants. Experiments on WebArena and AndroidWorld show that UI-Simulator rivals or surpasses open-source agents trained on real UIs with significantly better robustness, despite using weaker teacher models. Moreover, UI-Simulator-Grow matches the performance of Llama-3-70B-Instruct using only Llama-3-8B-Instruct as the base model, highlighting the potential of targeted synthesis scaling paradigm to continuously and efficiently enhance the digital agents.
Paper Structure (48 sections, 6 equations, 8 figures, 21 tables)

This paper contains 48 sections, 6 equations, 8 figures, 21 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. World Models.
  4. Synthetic Data for Digital Agent Training.
  5. Digital World Models in UI-Simulator
  6. Formulation
  7. (Retrieval-Free) Simulation
  8. Predict an Overview of the Next State.
  9. Generate Rich Draft in Natural Language.
  10. Convert Unstructured Draft into Structured Format.
  11. Retrieval-Augmented Simulation
  12. Scalable Synthetic Training Trajectory Collection in Simulated World
  13. Overview and Formulation
  14. Step-Wise Guided Rollout Process
  15. Step-Wise Task Control Proposal.
  16. ...and 33 more sections

Figures (8)

  • Figure 1: Overview and performance highlights of UI-Simulator and UI-Simulator-Grow.
  • Figure 2: Overall process of how the retrieval-free/-augmented simulators predict the next UI state.
  • Figure 3: The effect of standard scaling and UI-Simulator-Grow targeted scaling.
  • Figure 4: Successful task numbers across the 5 main task categories through the three iterations of the UI-Simulator-Grow scaling.
  • Figure 5: Illustration of overall target task selection process.
  • ...and 3 more figures