GAIA: A Data Flywheel System for Training GUI Test-Time Scaling Critic Models

TL;DR

GAIA addresses the risk of irreversible GUI actions by introducing a data-driven GUI Action Critic framework. It trains an Intuitive Critic Model (ICM) on real agent actions and leverages a data flywheel to iteratively refine the critic into ICM-r2, enabling more reliable test-time action selection via a Best-of-N rollout. The approach yields significant improvements in planning and grounding across both open-source and closed-source GUI agents and generalizes to unseen models, all without retraining the base agents. By combining real-action data with a binary critique and iterative data augmentation, GAIA offers a scalable path to more robust GUI automation.

Abstract

While Large Vision-Language Models (LVLMs) have significantly advanced GUI agents' capabilities in parsing textual instructions, interpreting screen content, and executing tasks, a critical challenge persists: the irreversibility of agent operations, where a single erroneous action can trigger catastrophic deviations. To address this, we propose the GUI Action Critic's Data Flywheel System (GAIA), a training framework that enables the models to have iterative critic capabilities, which are used to improve the Test-Time Scaling (TTS) of basic GUI agents' performance. Specifically, we train an Intuitive Critic Model (ICM) using positive and negative action examples from a base agent first. This critic evaluates the immediate correctness of the agent's intended actions, thereby selecting operations with higher success probability. Then, the initial critic guides agent actions to collect refined positive/negative samples, initiating the self-improving cycle. The augmented data then trains a second-round critic with enhanced discernment capability. We conduct experiments on various datasets and demonstrate that the proposed ICM can improve the test-time performance of various closed-source and open-source models, and the performance can be gradually improved as the data is recycled. The code and dataset will be publicly released.

Figures (8)

• Figure 1: Intuitive results. ICM guides agents' action during testing, as shown in (a), thereby improving the agent's accuracy, and is continuously improved by the data flywheel, as shown in (b).
• Figure 2: Data flywheel curation pipeline for GAIA. A sample dataset is constructed using GUI agent interactions. The positive and negative labels are marked by comparing the ground truth actions to train an action correctness discrimination model. After the critic model guides the GUI agent, it further expands the dataset, pushing the data flywheel to cover more action distributions, thereby promoting the iterative improvement of model performance.
• Figure 3: Test-Time scaling pipeline. Through best-of-N rollout, multi-candidate actions of GUI agents are given, and the correct action with the highest probability is selected after ICM evaluation.
• Figure 4: Performance improvements of Pass@N and N-rollout.
• Figure 5: Visualization result. The basic agent selects the wrong action. Based on the action rollout, both ICM and ICM-r2 select the correct action from the candidates.