VideoVerse: How Far is Your T2V Generator from a World Model?

TL;DR

VideoVerse introduces a world-model-aware benchmark for Text-to-Video systems, combining static and dynamic evaluation dimensions into 300 prompts that cover 815 events and 793 binary questions. It couples a prompt-construction pipeline with hidden-semantics design and a two-pronged evaluation using Longest Common Subsequence for temporal causality and VLM-based binary inquiries, aiming to quantify how close current T2V models are to true world models. Empirical results show clear gaps between open-source and closed-source models, with Veo-3 attaining the best overall performance but still falling short of world-model capabilities. The work also validates a human-preference-aligned evaluation pipeline via a user study, suggesting VideoVerse as a practical platform to guide future development toward more robust world-model T2V systems.

Abstract

The recent rapid advancement of Text-to-Video (T2V) generation technologies, which are critical to build ``world models'', makes the existing benchmarks increasingly insufficient to evaluate state-of-the-art T2V models. First, current evaluation dimensions, such as per-frame aesthetic quality and temporal consistency, are no longer able to differentiate state-of-the-art T2V models. Second, event-level temporal causality, which not only distinguishes video from other modalities but also constitutes a crucial component of world models, is severely underexplored in existing benchmarks. Third, existing benchmarks lack a systematic assessment of world knowledge, which are essential capabilities for building world models. To address these issues, we introduce VideoVerse, a comprehensive benchmark that focuses on evaluating whether a T2V model could understand complex temporal causality and world knowledge in the real world. We collect representative videos across diverse domains (e.g., natural landscapes, sports, indoor scenes, science fiction, chemical and physical experiments) and extract their event-level descriptions with inherent temporal causality, which are then rewritten into text-to-video prompts by independent annotators. For each prompt, we design a suite of binary evaluation questions from the perspective of dynamic and static properties, with a total of ten carefully defined evaluation dimensions. In total, our VideoVerse comprises 300 carefully curated prompts, involving 815 events and 793 binary evaluation questions. Consequently, a human preference aligned QA-based evaluation pipeline is developed by using modern vision-language models. Finally, we perform a systematic evaluation of state-of-the-art open-source and closed-source T2V models on VideoVerse, providing in-depth analysis on how far the current T2V generators are from world models.

Figures (8)

• Figure 1: Overview of the evaluation dimensions of VideoVerse, which are considered from two complementary perspectives: the Dynamic and the Static. Under the two categories, a total of ten dimensions, which include six world model level evaluation dimensions and four basic level evaluation dimensions.
• Figure 2: Left: We extract CLIP embeddings of prompts from mainstream T2V benchmarks and compute their cosine similarity. We see that existing benchmarks contain a large number of redundant prompts. Middle: Users typically provide complex instructions when interacting with world model level T2V systems, yet existing benchmarks generally consist of overly short prompts. Right: The prompt length distribution of VideoVerse aligns closely with natural usage patterns. We also provide more comparisons with other T2V benchmarks in the Appendix \ref{['appendix:Details_Statistics_of_Our_VideoVerse']}.
• Figure 3: Case study of T2V models' performance on our VideoVerse. Gemini 2.5 Pro is used as the evaluator. Wan 2.1 and Hunyuan successfully generate the corresponding attribution content (horse's coat glistens) but struggle with Event Following and Common Sense, whereas Veo-3 demonstrates strong performance across all dimensions.
• Figure 4: Prompt construction pipeline of VideoVerse. Source prompts are drawn from three domains: science fiction (VidProM), daily life (ActivityNet), and human collected high school level experiments. After domain specific filtering, GPT-4o extracts temporally related events to form raw prompts. Independent annotators then refine these raw prompts by incorporating one or more evaluation dimensions, while preserving the original event structure, to produce the final prompts.
• Figure 5: Comparison of scene uniqueness across T2V benchmarks. Scenes implied by prompts are extracted using GPT-4o, and semantic embeddings are used to merge similar scenes. VideoVerse achieves the highest uniqueness, ensuring broader and more diverse scene coverage.