EmbodiedVSR: Dynamic Scene Graph-Guided Chain-of-Thought Reasoning for Visual Spatial Tasks

TL;DR

This work targets spatial reasoning in embodied AI, where current multimodal language models often hallucinate spatial relations and struggle with long-horizon planning. It introduces EmbodiedVSR, a zero-shot framework that fuses dynamic scene graphs with physics-constrained Chain-of-Thought reasoning, grounded in action-conditioned environmental dynamics. The authors also present the eSpatial-Benchmark, comprising eSpatial-X, eSpatial-RoboMIND, and eSpatial-Lego, to rigorously evaluate embodied spatial capabilities in both simulated and real-world settings. Empirical results demonstrate improved accuracy and reasoning coherence over baseline MLLMs, including successful real-world LEGO block reassembly, underscoring the potential of structured grounding for reliable embodied spatial intelligence.

Abstract

While multimodal large language models (MLLMs) have made groundbreaking progress in embodied intelligence, they still face significant challenges in spatial reasoning for complex long-horizon tasks. To address this gap, we propose EmbodiedVSR (Embodied Visual Spatial Reasoning), a novel framework that integrates dynamic scene graph-guided Chain-of-Thought (CoT) reasoning to enhance spatial understanding for embodied agents. By explicitly constructing structured knowledge representations through dynamic scene graphs, our method enables zero-shot spatial reasoning without task-specific fine-tuning. This approach not only disentangles intricate spatial relationships but also aligns reasoning steps with actionable environmental dynamics. To rigorously evaluate performance, we introduce the eSpatial-Benchmark, a comprehensive dataset including real-world embodied scenarios with fine-grained spatial annotations and adaptive task difficulty levels. Experiments demonstrate that our framework significantly outperforms existing MLLM-based methods in accuracy and reasoning coherence, particularly in long-horizon tasks requiring iterative environment interaction. The results reveal the untapped potential of MLLMs for embodied intelligence when equipped with structured, explainable reasoning mechanisms, paving the way for more reliable deployment in real-world spatial applications. The codes and datasets will be released soon.

Figures (6)

• Figure 1: Overview of EmbodiedVSR, a framework integrating multimodal interaction and dynamic task execution. EmbodiedVSR fuses dynamic scene graph generation, embodied spatial chain-of-thought reasoning and robotic control, enabling robots to grasp spatial object relationships. Additionally, we developed the eSpatial-Benchmark and dataset, further empowering the application of multimodal large models in embodied intelligence scenarios.
• Figure 2: eSpatial datasets data distributions.
• Figure 3: eSpatial filtering process.
• Figure 4: eSpatial-RoboMIND Benchmark evaluation
• Figure 5: LEGO block reassembly task setup. This task requires a humanoid with a two - finger gripper to reassemble a LEGO structure according to a given example. An additional camera is used to feed data to the EmbodiedVSR. A LangGraph - based agent formulates the outputs of EmbodiedVSR as actionable commands for the robot controller.