Visual Generation Unlocks Human-Like Reasoning through Multimodal World Models

TL;DR

This work addresses the gap between human-like reasoning and current AI in physically grounded tasks by formalizing a world-model perspective that jointly leverages verbal and visual representations. It introduces a principled framework tying world reconstruction and world simulation to chain-of-thought reasoning, and proposes VisWorld-Eval to evaluate reasoning that explicitly uses visual world modeling. Empirical results with a state-of-the-art Unified Multimodal Model show that interleaved verbal-visual CoT improves performance on tasks requiring visual world modeling (e.g., paper folding, ball tracking, cube views) and yields higher fidelity internal representations, while offering limited gains on tasks that do not demand visual modeling (e.g., mazes). The study demonstrates that visual generation can unlock more human-like multimodal reasoning, provides a benchmark for future research, and highlights the need for modality-aligned pre-training and RL methods to fully realize the potential of multimodal world models.

Abstract

Humans construct internal world models and reason by manipulating the concepts within these models. Recent advances in AI, particularly chain-of-thought (CoT) reasoning, approximate such human cognitive abilities, where world models are believed to be embedded within large language models. Expert-level performance in formal and abstract domains such as mathematics and programming has been achieved in current systems by relying predominantly on verbal reasoning. However, they still lag far behind humans in domains like physical and spatial intelligence, which require richer representations and prior knowledge. The emergence of unified multimodal models (UMMs) capable of both verbal and visual generation has therefore sparked interest in more human-like reasoning grounded in complementary multimodal pathways, though their benefits remain unclear. From a world-model perspective, this paper presents the first principled study of when and how visual generation benefits reasoning. Our key position is the visual superiority hypothesis: for certain tasks--particularly those grounded in the physical world--visual generation more naturally serves as world models, whereas purely verbal world models encounter bottlenecks arising from representational limitations or insufficient prior knowledge. Theoretically, we formalize internal world modeling as a core component of CoT reasoning and analyze distinctions among different forms of world models. Empirically, we identify tasks that necessitate interleaved visual-verbal CoT reasoning, constructing a new evaluation suite, VisWorld-Eval. Controlled experiments on a state-of-the-art UMM show that interleaved CoT significantly outperforms purely verbal CoT on tasks that favor visual world modeling, but offers no clear advantage otherwise. Together, this work clarifies the potential of multimodal world modeling for more powerful, human-like multimodal AI.

Figures (19)

• Figure 1: Overview of a world-model perspective on multimodal reasoning. (a) Humans construct mental models of the world, representing information and knowledge through two complementary channels--verbal and visual--to support reasoning, planning, and decision-making. (b) Recent advances in large language models (LLMs) and vision language models (VLMs) largely rely on verbal chain-of-thought reasoning, leveraging primarily verbal and symbolic world knowledge. (c) Unified multimodal models (UMMs) open a new paradigm by using visual generation for visual world modeling, advancing more human-like reasoning on tasks grounded in the physical world. Examples of reasoning with verbal world modeling are adapted from guo2025deepseekdu2025revisitingchen2025planningzhang2025agent.
• Figure 2: Theoretical formulation of the world model perspective on multimodal reasoning. (a) Observations of the same underlying world state can span multiple modalities, including verbal and visual observations, each reflecting different views or emphases. (b) Two atomic capabilities of world models are defined: world reconstruction, which infers complete structure from partial observations and enables novel view synthesis, and world simulation, which models dynamics to predict future observations. (c) Chain-of-thought reasoning includes internal world modeling, by explicitly maintaining an evolving sequence of observations, generated through either of the atomic world model capabilities.
• Figure 3: The VisWorld-Eval suite for assessing multimodal reasoning with visual world modeling. VisWorld-Eval comprises seven tasks spanning both synthetic and real-world domains, each designed to isolate and demand specific atomic world-model capabilities.
• Figure 4: Performance of SFT-trained UMMs with different world model-based chain-of-thought formulations across seven tasks from VisWorld-Eval. Refer to Table \ref{['tab:leaderboard']} for zero-shot performance of advanced VLMs.
• Figure 5: Probing implicit world models, by training a set of probes, i.e., MLPs which infer the masked point coordinates during reasoning from internal representations.

Theorems & Definitions (18)

• Theorem 1
• Theorem 2
• Theorem 3: Restatement of Theorem \ref{['thm:kl_chain_rule']}
• proof
• Theorem 4: Restatement of Theorem \ref{['thm:mutual_info']}
• proof
• Corollary 1
• proof
• Lemma 1: Uniform Loss Shift under Total Variation
• proof