CausalSpatial: A Benchmark for Object-Centric Causal Spatial Reasoning

TL;DR

CausalSpatial defines a novel benchmark to evaluate object-centric causal spatial reasoning across four tasks (Collision, Compatibility, Occlusion, Trajectory) and reveals a substantial gap between human and state-of-the-art multimodal LLMs, largely due to reliance on ungrounded textual Chain-of-Thought. To address this, the authors propose Causal Object World Model (COW), an object-centric world model that externalizes dynamics by generating trajectory-controlled videos, providing explicit visual grounding to support temporally consistent reasoning. Across extensive evaluations, COW yields measurable improvements in collision and trajectory prediction by grounding the model in physically plausible visuals, though challenges remain for long-horizon occlusion and accurate 3D parameter estimation. The work contributes both a public dataset/code and a pathway toward integrating explicit visual simulations with language models to advance robust, grounded causal spatial reasoning in AI systems.

Abstract

Humans can look at a static scene and instantly predict what happens next -- will moving this object cause a collision? We call this ability Causal Spatial Reasoning. However, current multimodal large language models (MLLMs) cannot do this, as they remain largely restricted to static spatial perception, struggling to answer "what-if" questions in a 3D scene. We introduce CausalSpatial, a diagnostic benchmark evaluating whether models can anticipate consequences of object motions across four tasks: Collision, Compatibility, Occlusion, and Trajectory. Results expose a severe gap: humans score 84% while GPT-5 achieves only 54%. Why do MLLMs fail? Our analysis uncovers a fundamental deficiency: models over-rely on textual chain-of-thought reasoning that drifts from visual evidence, producing fluent but spatially ungrounded hallucinations. To address this, we propose the Causal Object World model (COW), a framework that externalizes the simulation process by generating videos of hypothetical dynamics. With explicit visual cues of causality, COW enables models to ground their reasoning in physical reality rather than linguistic priors. We make the dataset and code publicly available here: https://github.com/CausalSpatial/CausalSpatial

Figures (14)

• Figure 1: Causal spatial reasoning task. The task requires model to progress from 2D image understanding to 3D spatial perception and ultimately to 4D causal spatial reasoning—predicting how actions affect future spatial configurations. Current MLLMs fail at this complex task due to hallucinations.
• Figure 2: CausalSpatial tasks. CausalSpatial encompasses four causal reasoning tasks that require models to anticipate physical outcomes in 3D scenes: Collision, Occlusion, Trajectory, and Compatibility. The number of evaluation entries for each task is listed in the middle. Each task is designed with two difficulty levels, denoted as "L1/L2". All scenes are rendered in Blender to provide realistic 3D environments for physics-grounded evaluation.
• Figure 3: Statistics for output length (token), True Positive Rate, and Not Sure Rate. The comparison emphasizes limited returns from model scaling on causal spatial reasoning, while larger models exhibit significantly lower NSR.
• Figure 4: Why do SoTA MLLMs fail in CausalSpatial? These examples illustrate how MLLMs often produce lengthy and seemingly coherent explanations while failing to ground their reasoning in the visual evidence. In the car–cabinet scenario (Left), the model ignores the visible placement of the toy car directly blocking the wooden calendar and instead follows a generic linguistic pattern about object removal. In the banana–box example (Right), the model asserts that the banana exceeds the box depth—an inference inconsistent with the image. Across such failure modes, the reasoning chains are verbose but inaccurate, revealing that current MLLMs fail to simulate the motion process by solely relying on the textual reasoning.
• Figure 5: (A) COW pipeline overview: COW is a 4D trajectory-controlled video generation method that enhances spatial reasoning by explicitly rendering object dynamics through object-centric video generation. (B) Qualitative results: COW produces physically and visually plausible simulation videos that offer clear visual cues for improved causal spatial inference.