MobileGUI-RL: Advancing Mobile GUI Agent through Reinforcement Learning in Online Environment

TL;DR

MobileGUI-RL tackles the brittleness of offline GUI agents by training in an online environment that supports continuous interaction with mobile UIs. It introduces a synthetic task curriculum generated via self-exploration and filtered with a text-based world model, coupled with MobGRPO, a trajectory-aware reinforcement learning method with multi-component rewards. Across three online mobile GUI benchmarks, fine-tuned 7B and 32B base models show consistent gains, with MobileGUI-32B outperforming larger baselines and rivaling closed-source leaders on AndroidWorld. The framework demonstrates that online trajectory-level feedback paired with curriculum learning significantly improves generalization and efficiency in mobile GUI navigation tasks.

Abstract

Recently, there has been a surge of vision-based GUI agents designed to automate everyday mobile and web tasks. These agents interpret raw GUI screenshots and autonomously decide where to click, scroll, or type, which bypasses handcrafted rules and app-specific APIs. However, most existing methods trained GUI agent in the offline environment using pre-collected trajectories. This approach limits scalability, causes overfitting to specific UI templates, and leads to brittle policies when faced with unseen environment. We present MobileGUI-RL, a scalable framework that trains GUI agent in online environment. MobileGUI-RL contains two key components. It (i) synthesizes a curriculum of learnable tasks through self-exploration and filtering, and (ii) adapts GRPO to GUI navigation with trajectory-aware advantages and composite rewards that balance task success and execution efficiency. Experiments on three online mobile-agent benchmarks show consistent gains, validating the effectiveness of our approach.

Figures (4)

• Figure 1: Framework overall -- a scalable pipeline for training GUI agents through self-exploration, task filtering, and trajectory-level reinforcement learning with a structured reward design.
• Figure 2: Training dynamics with and without curriculum learning for 7B and 32B models. The top row shows the mean trajectory base reward, and the bottom row shows the ratio of impossible tasks encountered. With curriculum learning (red), the reward first rises on easy tasks and then falls as the curriculum introduces harder tasks, which is corroborated by the rising impossible task ratio. This structured approach leads to better final performance than training without a curriculum (blue).
• Figure 3: Case Studies. The case illustrates the task: "Create a calendar event for tomorrow at 20h with the title 'Call with the Team' and the description 'We will prepare for team roles.'. The event should last for 30 mins." The left shows the execution before reinforcement learning, while the right shows the result after RL (ours). The pre-RL agent misses two critical steps: (1) omitting the meeting description, and (2) failing to set the event’s end time.
• Figure :