Scalable Policy Evaluation with Video World Models

TL;DR

The paper tackles the expensive and unsafe nature of real-world evaluation for generalist robot manipulation policies by introducing action-conditional video prediction as a scalable world model for policy assessment. It extends a large pre-trained diffusion video model (Cosmos-Predict2-Video2World-2B) with action conditioning and uses a Vision-Language Model to judge task success in rollout videos, enabling automated evaluation without physical rollouts. Through extensive synthetic and real-world experiments, it analyzes data augmentation, pretraining, and backbone choices, showing that the approach yields meaningful policy rankings and correlations with ground-truth performance, albeit with identifiable failure modes. The work demonstrates a practical, data-driven framework for benchmarking and accelerating progress toward robust, generalist robotic policies while reducing reliance on costly real-world experimentation.

Abstract

Training generalist policies for robotic manipulation has shown great promise, as they enable language-conditioned, multi-task behaviors across diverse scenarios. However, evaluating these policies remains difficult because real-world testing is expensive, time-consuming, and labor-intensive. It also requires frequent environment resets and carries safety risks when deploying unproven policies on physical robots. Manually creating and populating simulation environments with assets for robotic manipulation has not addressed these issues, primarily due to the significant engineering effort required and the substantial sim-to-real gap, both in terms of physics and rendering. In this paper, we explore the use of action-conditional video generation models as a scalable way to learn world models for policy evaluation. We demonstrate how to incorporate action conditioning into existing pre-trained video generation models. This allows leveraging internet-scale in-the-wild online videos during the pre-training stage and alleviates the need for a large dataset of paired video-action data, which is expensive to collect for robotic manipulation. Our paper examines the effect of dataset diversity, pre-trained weights, and common failure cases for the proposed evaluation pipeline. Our experiments demonstrate that across various metrics, including policy ranking and the correlation between actual policy values and predicted policy values, these models offer a promising approach for evaluating policies without requiring real-world interactions.

Figures (8)

• Figure 1: Overview. We propose using action-conditional video prediction models as world models for scalable policy evaluation. The process involves: (a) deploying a policy into the video model to generate rollout videos conditioned on action sequences and initial frame, (b) using a VLM to judge task success or failure, and (c) computing the correlation between predicted and actual policy performance to evaluate the reliability of the approach.
• Figure 2: Network Architecture for Action-Conditional Video Prediction with a DiT-Based Video Model such as Cosmos-Predict2-2B nvidia_cosmos_predict2. We process the raw action information with Fourier feature mappings fourier and encode it into a latent action token. The latent action token is added to a time-embedding, which enables us to make the video diffusion model action-conditional.
• Figure 3: Correlation Plots for Policy Evaluation. We plot the correlation between predicted policy performance via evaluation in the video model vs. actual policy performance for the simulation tasks.
• Figure 4: Manipulation Tasks. We evaluate the proposed approach in both synthetic and real-world tasks. In the synthetic setting, we mainly consider four tasks in RoboMimic corl21_robomimic: lift, can, square and tool hang. For real-world experiments, we consider four tasks from the Bridge benchmark: lifting a carrot, a pot, an eggplant, and a cup.
• Figure 5: Effect of Policy Rollouts on Policy Evaluation. (a) Without policy rollouts, predictions degrade over time—for example, the gripper gradually disappears. Incorporating rollouts significantly improves visual quality and realism. (b) Increasing the number of policy rollouts in the video model’s training set consistently enhances policy evaluation performance.