4DWorldBench: A Comprehensive Evaluation Framework for 3D/4D World Generation Models

TL;DR

4DWorldBench introduces a unified, multimodal, physics-aware benchmark for evaluating next-generation 3D/4D world generation models. It jointly measures perceptual quality, 4D condition alignment, physical realism, and 4D consistency using a hybrid evaluation framework that blends MLLM-based grounding with LLM-based abstract reasoning, applicable to text-, image-, and video-conditioned generation. The framework provides a curated dataset, adaptive conditioning, and a leaderboard, and demonstrates that adaptive, dimension-aware QA improves alignment with human judgments while revealing modality gaps in physical realism and motion. The work offers a scalable path toward coherent, controllable, and physically plausible world generation and outlines directions for broader real-world evaluation and lightweight deployment.

Abstract

World Generation Models are emerging as a cornerstone of next-generation multimodal intelligence systems. Unlike traditional 2D visual generation, World Models aim to construct realistic, dynamic, and physically consistent 3D/4D worlds from images, videos, or text. These models not only need to produce high-fidelity visual content but also maintain coherence across space, time, physics, and instruction control, enabling applications in virtual reality, autonomous driving, embodied intelligence, and content creation. However, prior benchmarks emphasize different evaluation dimensions and lack a unified assessment of world-realism capability. To systematically evaluate World Models, we introduce the 4DWorldBench, which measures models across four key dimensions: Perceptual Quality, Condition-4D Alignment, Physical Realism, and 4D Consistency. The benchmark covers tasks such as Image-to-3D/4D, Video-to-4D, Text-to-3D/4D. Beyond these, we innovatively introduce adaptive conditioning across multiple modalities, which not only integrates but also extends traditional evaluation paradigms. To accommodate different modality-conditioned inputs, we map all modality conditions into a unified textual space during evaluation, and further integrate LLM-as-judge, MLLM-as-judge, and traditional network-based methods. This unified and adaptive design enables more comprehensive and consistent evaluation of alignment, physical realism, and cross-modal coherence. Preliminary human studies further demonstrate that our adaptive tool selection achieves closer agreement with subjective human judgments. We hope this benchmark will serve as a foundation for objective comparisons and improvements, accelerating the transition from "visual generation" to "world generation." Our project can be found at https://yeppp27.github.io/4DWorldBench.github.io/.

Figures (8)

• Figure 1: Overview of the 4DWorldBench framework. The benchmark evaluates generation quality across four key dimensions: 4D consistency, condition alignment, perceptual quality, and physical realism. It supports diverse generative settings, including text-, image-, and video-to-3D/4D generation. The framework integrates hybrid evaluation metrics—model-based scores, feature-based similarity, LLM-based QA, and MLLM-based QA—together with human studies for reliability. Condition- and dimension-adaptive QA modules leverage MLLMs for concrete visual grounding and LLMs for higher-level physical reasoning.
• Figure 2: Text Condition Distribution.
• Figure 3: Image Condition Distribution.
• Figure 4: Video Condition Distribution.
• Figure 5: Pipeline of the Physical Realism Evaluation. The framework first unifies text, image, and video conditions into a common textual form, then uses an LLM to adaptively select relevant physical dimensions and generate diagnostic questions. An LLM-based QA module compares predicted and reference answers to yield a continuous physical realism score.