Anticipatory Planning for Multimodal AI Agents

Abstract

Recent advances in multimodal agents have improved computer-use interaction and tool-usage, yet most existing systems remain reactive, optimizing actions in isolation without reasoning about future states or long-term goals. This limits planning coherence and prevents agents from reliably solving high-level, multi-step tasks. We introduce TraceR1, a two-stage reinforcement learning framework that explicitly trains anticipatory reasoning by forecasting short-horizon trajectories before execution. The first stage performs trajectory-level reinforcement learning with rewards that enforce global consistency across predicted action sequences. The second stage applies grounded reinforcement fine-tuning, using execution feedback from frozen tool agents to refine step-level accuracy and executability. TraceR1 is evaluated across seven benchmarks, covering online computer-use, offline computer-use benchmarks, and multimodal tool-use reasoning tasks, where it achieves substantial improvements in planning stability, execution robustness, and generalization over reactive and single-stage baselines. These results show that anticipatory trajectory reasoning is a key principle for building multimodal agents that can reason, plan, and act effectively in complex real-world environments.

Figures (5)

• Figure 1: TraceR1 overview: anticipatory planning and grounded execution. The planning model takes a user instruction together with the current screenshot and interaction history, predicts a trajectory of future actions, where only the first predicted step is executed by the tool agent while later steps are unexecuted lookahead predictions, and generates step-level instructions for tool agents to complete tasks across diverse GUI environments and tool-use platforms.
• Figure 2: Two-stage training framework of TraceR1. Stage $1$ performs anticipatory trajectory optimization using trajectory-level alignment rewards, while Stage $2$ applies grounded RL fine-tuning with step-level rewards derived from tool-agent execution feedback.
• Figure 3: Two-stage Training Ablation. Performance (%) comparison on AndroidWorld, OSWorld-Verified and GTA benchmarks. Stage $2$ provides consistent improvements in both settings.
• Figure 4: Example trajectory: coordination between TraceR1 (planner) and UI-TARS-1.5-7B (executor) on OSWorld-Verified.
• Figure 5: Ablation results. (a) shows the effect of predictive trajectory horizon length on AndroidWorld. (b--c) show the impact of removing $\lambda_{\text{rep}}$ and $\gamma$ during training on AndroidControl-High and GTA.