Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Video-Based Reward Modeling for Computer-Use Agents

Linxin Song, Jieyu Zhang, Huanxin Sheng, Taiwei Shi, Gupta Rahul, Yang Liu, Ranjay Krishna, Jian Kang, Jieyu Zhao

TL;DR

Results show that video-execution reward modeling can serve as a scalable, model-agnostic evaluator for CUAs, and design spatiotemporal token pruning, which removes homogeneous regions and persistent tokens while preserving decisive UI changes.

Abstract

Computer-using agents (CUAs) are becoming increasingly capable; however, it remains difficult to scale evaluation of whether a trajectory truly fulfills a user instruction. In this work, we study reward modeling from execution video: a sequence of keyframes from an agent trajectory that is independent of the agent's internal reasoning or actions. Although video-execution modeling is method-agnostic, it presents key challenges, including highly redundant layouts and subtle, localized cues that determine success. We introduce Execution Video Reward 53k (ExeVR-53k), a dataset of 53k high-quality video--task--reward triplets. We further propose adversarial instruction translation to synthesize negative samples with step-level annotations. To enable learning from long, high-resolution execution videos, we design spatiotemporal token pruning, which removes homogeneous regions and persistent tokens while preserving decisive UI changes. Building on these components, we fine-tune an Execution Video Reward Model (ExeVRM) that takes only a user instruction and a video-execution sequence to predict task success. Our ExeVRM 8B achieves 84.7% accuracy and 87.7% recall on video-execution assessment, outperforming strong proprietary models such as GPT-5.2 and Gemini-3 Pro across Ubuntu, macOS, Windows, and Android, while providing more precise temporal attribution. These results show that video-execution reward modeling can serve as a scalable, model-agnostic evaluator for CUAs.

Video-Based Reward Modeling for Computer-Use Agents

TL;DR

Results show that video-execution reward modeling can serve as a scalable, model-agnostic evaluator for CUAs, and design spatiotemporal token pruning, which removes homogeneous regions and persistent tokens while preserving decisive UI changes.

Abstract

Computer-using agents (CUAs) are becoming increasingly capable; however, it remains difficult to scale evaluation of whether a trajectory truly fulfills a user instruction. In this work, we study reward modeling from execution video: a sequence of keyframes from an agent trajectory that is independent of the agent's internal reasoning or actions. Although video-execution modeling is method-agnostic, it presents key challenges, including highly redundant layouts and subtle, localized cues that determine success. We introduce Execution Video Reward 53k (ExeVR-53k), a dataset of 53k high-quality video--task--reward triplets. We further propose adversarial instruction translation to synthesize negative samples with step-level annotations. To enable learning from long, high-resolution execution videos, we design spatiotemporal token pruning, which removes homogeneous regions and persistent tokens while preserving decisive UI changes. Building on these components, we fine-tune an Execution Video Reward Model (ExeVRM) that takes only a user instruction and a video-execution sequence to predict task success. Our ExeVRM 8B achieves 84.7% accuracy and 87.7% recall on video-execution assessment, outperforming strong proprietary models such as GPT-5.2 and Gemini-3 Pro across Ubuntu, macOS, Windows, and Android, while providing more precise temporal attribution. These results show that video-execution reward modeling can serve as a scalable, model-agnostic evaluator for CUAs.
Paper Structure (31 sections, 8 equations, 11 figures, 5 tables, 3 algorithms)

This paper contains 31 sections, 8 equations, 11 figures, 5 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Reward Evaluation for Visual GUI Agents
  4. Efficient Video Understanding and Token Pruning
  5. Execution Video Reward Modeling
  6. ExeVR-53k
  7. OSWorld
  8. AgentNet
  9. ScaleCUA
  10. Adversarial Instruction Translation
  11. Spatiotemporal Token Pruning
  12. Spatial Token Pruning
  13. Temporal Token Pruning
  14. Experiments
  15. Implementation Details
  16. ...and 16 more sections

Figures (11)

  • Figure 1: Task distribution of ExeVR-53k.
  • Figure 2: Illustration of how we synthesize negative samples via adversarial instruction translation. We use GPT-5.2 as the Vision Language Model.
  • Figure 3: Comparison of temporal IoU (tIoU) scores across models on ExeVR-Bench.
  • Figure 4: Efficiency analysis. Left: memory usage. Right: runtime.
  • Figure 5: Comparison of STP and TTP
  • ...and 6 more figures