Agent Alpha: Tree Search Unifying Generation, Exploration and Evaluation for Computer-Use Agents

TL;DR

Agent Alpha introduces a unified, step-level Monte Carlo Tree Search framework that synergizes generation, exploration, and evaluation for computer-use agents. By employing the Alpha-UCT bound, tree-informed reflections, and diversity-aware expansion, it enables regressive planning and efficient prefix reuse in large GUI action spaces. The method advances three design pillars—tree-informed action generation, diversity-constrained exploration, and comparison-driven evaluation—and provides a regret-analysis framing for its bound. Empirically, it achieves state-of-the-art performance on the OSWorld benchmark and demonstrates robust ablations and hyperparameter insights, highlighting practical improvements in accuracy and efficiency for multimodal GUI control tasks.

Abstract

While scaling test-time compute through trajectory-level sampling has significantly improved Graphical User Interface (GUI) agents, the lack of regressive ability prevents the reuse of partial successes and the recovery from early missteps. In this paper, we introduce Agent Alpha, a unified framework that synergizes generation, exploration, and evaluation through step-level Monte Carlo Tree Search (MCTS). It enables active modeling or exploiting structures of the planning space. By integrating alpha-UCT guided search into the interaction loop, Agent Alpha enables deliberate planning, facilitating early pruning of suboptimal branches and efficient prefix reuse. We also employ comparison-driven evaluation to mitigate absolute scoring biases and diversity-constrained expansion to maintain a compact, informative search space. Regret bound of alpha-UCT is analyzed. On the OSWorld benchmark, Agent Alpha achieves a state-of-the-art success rate of $\sim 77\%$, significantly outperforming trajectory-level baselines under equivalent compute.

Figures (7)

• Figure 1: Overview of Agent Alpha. Agent Alpha could provide regressive planning that improve the utilization of generated trajectories with refined exploration and evaluation. We use nodes with deeper color to represent states with higher potential. $\mathcal{T}, \mathcal{R}, \mathcal{I}, \mathcal{O}, \mathcal{C}$ denotes the information of trajectory, reflection, task instruction, observation, and operation context respectively. Existing approaches operate primarily as unidirectional processes without the ability to model or exploit structures of the planning space in complex dynamic environments. Agent Alpha addresses this by synergizing generation, exploration, and evaluation capabilities of MLLMs through step-level MCTS under Alpha-UCT bound.
• Figure 2: The detailed search process applied in Agent Alpha. The process consists of Selection, Expansion, Evaluation, and Back-Propagation. The search process will stop if the budget runs out or the task is successful.
• Figure 3: Analysis of hyperparameters: (a) Success rate vs. Expansion Nodes, (b) Success rate vs. Maximum Iterations, (c) Max Depth vs. Expansion factor, and (d) Success rate vs. Action chunking on long-horizon tasks.
• Figure 4: MCTS Tree Structure for Task2(Optimal Path Highlighted)
• Figure 5: Trajectory Visualization of Task2: Part 1

Theorems & Definitions (3)

• Theorem 1
• Corollary 2
• Corollary 3