Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

World Reasoning Arena

PAN Team, Qiyue Gao, Kun Zhou, Jiannan Xiang, Zihan Liu, Dequan Yang, Junrong Chen, Arif Ahmad, Cong Zeng, Ganesh Bannur, Xinqi Huang, Zheqi Liu, Yi Gu, Yichi Yang, Guangyi Liu, Zhiting Hu, Zhengzhong Liu, Eric Xing

Abstract

World models (WMs) are intended to serve as internal simulators of the real world that enable agents to understand, anticipate, and act upon complex environments. Existing WM benchmarks remain narrowly focused on next-state prediction and visual fidelity, overlooking the richer simulation capabilities required for intelligent behavior. To address this gap, we introduce WR-Arena, a comprehensive benchmark for evaluating WMs along three fundamental dimensions of next world simulation: (i) Action Simulation Fidelity, the ability to interpret and follow semantically meaningful, multi-step instructions and generate diverse counterfactual rollouts; (ii) Long-horizon Forecast, the ability to sustain accurate, coherent, and physically plausible simulations across extended interactions; and (iii) Simulative Reasoning and Planning, the ability to support goal-directed reasoning by simulating, comparing, and selecting among alternative futures in both structured and open-ended environments. We build a task taxonomy and curate diverse datasets designed to probe these capabilities, moving beyond single-turn and perceptual evaluations. Through extensive experiments with state-of-the-art WMs, our results expose a substantial gap between current models and human-level hypothetical reasoning, and establish WR-Arena as both a diagnostic tool and a guideline for advancing next-generation world models capable of robust understanding, forecasting, and purposeful action. The code is available at https://github.com/MBZUAI-IFM/WR-Arena.

World Reasoning Arena

Abstract

World models (WMs) are intended to serve as internal simulators of the real world that enable agents to understand, anticipate, and act upon complex environments. Existing WM benchmarks remain narrowly focused on next-state prediction and visual fidelity, overlooking the richer simulation capabilities required for intelligent behavior. To address this gap, we introduce WR-Arena, a comprehensive benchmark for evaluating WMs along three fundamental dimensions of next world simulation: (i) Action Simulation Fidelity, the ability to interpret and follow semantically meaningful, multi-step instructions and generate diverse counterfactual rollouts; (ii) Long-horizon Forecast, the ability to sustain accurate, coherent, and physically plausible simulations across extended interactions; and (iii) Simulative Reasoning and Planning, the ability to support goal-directed reasoning by simulating, comparing, and selecting among alternative futures in both structured and open-ended environments. We build a task taxonomy and curate diverse datasets designed to probe these capabilities, moving beyond single-turn and perceptual evaluations. Through extensive experiments with state-of-the-art WMs, our results expose a substantial gap between current models and human-level hypothetical reasoning, and establish WR-Arena as both a diagnostic tool and a guideline for advancing next-generation world models capable of robust understanding, forecasting, and purposeful action. The code is available at https://github.com/MBZUAI-IFM/WR-Arena.

Paper Structure

This paper contains 34 sections, 5 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. World Model
  4. Definition.
  5. World Model for Simulative Reasoning and Planning.
  6. World Model Evaluation
  7. Limitations of Existing Benchmarks.
  8. Our Focus.
  9. Evaluation Framework
  10. Action Simulation Fidelity
  11. Agent Simulation.
  12. Environment Simulation.
  13. Long-horizon Forecast
  14. Transition Smoothness.
  15. Generation Consistency.
  16. ...and 19 more sections

Figures (5)

  • Figure 1: The different focus of existing world model and our evaluation benchmarks.
  • Figure 2: Taxonomy with detailed examples of our evaluation benchmark.
  • Figure 3: The generation consistency curve w.r.t. the action sequence length. Eight world models or video generators execute identical 9-round action sequences from the same initial observations, with consistency scores measured at each round. Most models achieve lower than 75% consistency after round 5 or 6, indicating fundamental limitations for long-horizon planning applications.
  • Figure 4: Qualitative comparison on Structured Simulation and Planning. Starting from the same tabletop observation ($S_0$) with the goal of arranging all blocks into a line, PAN, Cosmos 1, and Cosmos 2 each simulate multi-step action plans through an iterative VLM–world model planning loop.
  • Figure 5: Qualitative comparison on Simulation Fidelity. Given a shared initial driving scene ($S_0$), PAN, Cosmos 1, and Cosmos 2 simulate a three-step sequence of environmental interventions from a dry forest road to light rain to intensifying rainfall.