Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Do World Action Models Generalize Better than VLAs? A Robustness Study

Zhanguang Zhang, Zhiyuan Li, Behnam Rahmati, Rui Heng Yang, Yintao Ma, Amir Rasouli, Sajjad Pakdamansavoji, Yangzheng Wu, Lingfeng Zhang, Tongtong Cao, Feng Wen, Xingyue Quan, Yingxue Zhang

Abstract

Robot action planning in the real world is challenging as it requires not only understanding the current state of the environment but also predicting how it will evolve in response to actions. Vision-language-action (VLA), which repurpose large-scale vision-language models for robot action generation using action experts, have achieved notable success across a variety of robotic tasks. Nevertheless, their performance remains constrained by the scope of their training data, exhibiting limited generalization to unseen scenarios and vulnerability to diverse contextual perturbations. More recently, world models have been revisited as an alternative to VLAs. These models, referred to as world action models (WAMs), are built upon world models that are trained on large corpora of video data to predict future states. With minor adaptations, their latent representation can be decoded into robot actions. It has been suggested that their explicit dynamic prediction capacity, combined with spatiotemporal priors acquired from web-scale video pretraining, enables WAMs to generalize more effectively than VLAs. In this paper, we conduct a comparative study of prominent state-of-the-art VLA policies and recently released WAMs. We evaluate their performance on the LIBERO-Plus and RoboTwin 2.0-Plus benchmarks under various visual and language perturbations. Our results show that WAMs achieve strong robustness, with LingBot-VA reaching 74.2% success rate on RoboTwin 2.0-Plus and Cosmos-Policy achieving 82.2% on LIBERO-Plus. While VLAs such as $π_{0.5}$ can achieve comparable robustness on certain tasks, they typically require extensive training with diverse robotic datasets and varied learning objectives. Hybrid approaches that partially incorporate video-based dynamic learning exhibit intermediate robustness, highlighting the importance of how video priors are integrated.

Do World Action Models Generalize Better than VLAs? A Robustness Study

Abstract

Robot action planning in the real world is challenging as it requires not only understanding the current state of the environment but also predicting how it will evolve in response to actions. Vision-language-action (VLA), which repurpose large-scale vision-language models for robot action generation using action experts, have achieved notable success across a variety of robotic tasks. Nevertheless, their performance remains constrained by the scope of their training data, exhibiting limited generalization to unseen scenarios and vulnerability to diverse contextual perturbations. More recently, world models have been revisited as an alternative to VLAs. These models, referred to as world action models (WAMs), are built upon world models that are trained on large corpora of video data to predict future states. With minor adaptations, their latent representation can be decoded into robot actions. It has been suggested that their explicit dynamic prediction capacity, combined with spatiotemporal priors acquired from web-scale video pretraining, enables WAMs to generalize more effectively than VLAs. In this paper, we conduct a comparative study of prominent state-of-the-art VLA policies and recently released WAMs. We evaluate their performance on the LIBERO-Plus and RoboTwin 2.0-Plus benchmarks under various visual and language perturbations. Our results show that WAMs achieve strong robustness, with LingBot-VA reaching 74.2% success rate on RoboTwin 2.0-Plus and Cosmos-Policy achieving 82.2% on LIBERO-Plus. While VLAs such as can achieve comparable robustness on certain tasks, they typically require extensive training with diverse robotic datasets and varied learning objectives. Hybrid approaches that partially incorporate video-based dynamic learning exhibit intermediate robustness, highlighting the importance of how video priors are integrated.
Paper Structure (26 sections, 3 figures, 9 tables)

This paper contains 26 sections, 3 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Vision Language Action (VLAs)
  4. World Model in Robotics
  5. World Models as Learned Simulators
  6. World Models as Auxiliary Tasks
  7. World Action Models (WAMs)
  8. Differences between WAMs and VLAs
  9. Experiments
  10. Datasets
  11. Evaluation Methods
  12. Results
  13. Conclusion
  14. RoboTwin 2.0-Plus Benchmark
  15. Perturbation Taxonomy
  16. ...and 11 more sections

Figures (3)

  • Figure 1: Examples of perturbations on RoboTwin 2.0-Plus tasks. Definition of notations (N1, L1, etc.) can be found in \ref{['app:robotwin_plus']}
  • Figure 2: Case studies on RoboTwin 2.0-Plus. We present three representative cases comparing $\pi_{0.5}$ and LingBot-VA. Keyframes are shown sequentially from left to right. (a)Task: beat block with hammer. Perturbation: noise (N3). Result:$\pi_{0.5}$ collides with the hammer and fails to complete the task. (b)Task: handover block. Perturbation: layout. Result:$\pi_{0.5}$ collides with the red block during approach, leading to failure. (c)Task: rank RGB blocks. Perturbation: lighting (L1–L4 mix). Result:$\pi_{0.5}$ fails to grasp the first red block due to misalignment and does not recover. LingBot-VA success in all three cases shown.
  • Figure 3: Illustrations of future images predicted by Cosmos-policy. Ground-truth and predicted images under three types of perturbations (noise, lighting, and background variations) in LIBERO-Plus are shown. GT: the ground truth. Pred.: the predictions generated by Cosmos-policy.