Emma-X: An Embodied Multimodal Action Model with Grounded Chain of Thought and Look-ahead Spatial Reasoning

TL;DR

Emma-X introduces a grounded, look-ahead-enabled Vision-Language-Action model for robotic manipulation. By constructing a hierarchical embodiment dataset from BridgeV2 and employing trajectory segmentation with HDBSCAN, it enables grounded chain-of-thought and predictive 3D/2D planning to guide action predictions. Empirical results on WidowX real-world tasks show substantial improvements over OpenVLA and ECoT, with strong gains in spatial-relations and OOD instructions, driven by segmentation, look-ahead planning, and grounded CoT. The work highlights the importance of visual-grounded reasoning for long-horizon robotic tasks and discusses latency and generalization as notable limitations, offering directions for future enhancement.

Abstract

Traditional reinforcement learning-based robotic control methods are often task-specific and fail to generalize across diverse environments or unseen objects and instructions. Visual Language Models (VLMs) demonstrate strong scene understanding and planning capabilities but lack the ability to generate actionable policies tailored to specific robotic embodiments. To address this, Visual-Language-Action (VLA) models have emerged, yet they face challenges in long-horizon spatial reasoning and grounded task planning. In this work, we propose the Embodied Multimodal Action Model with Grounded Chain of Thought and Look-ahead Spatial Reasoning, Emma-X. Emma-X leverages our constructed hierarchical embodiment dataset based on BridgeV2, containing 60,000 robot manipulation trajectories auto-annotated with grounded task reasoning and spatial guidance. Additionally, we introduce a trajectory segmentation strategy based on gripper states and motion trajectories, which can help mitigate hallucination in grounding subtask reasoning generation. Experimental results demonstrate that Emma-X achieves superior performance over competitive baselines, particularly in real-world robotic tasks requiring spatial reasoning.

Figures (5)

• Figure 1: Comparison of our Emma-X with ECoT in task reasoning. While both approaches utilize Gemini, our method also incorporates image sequence input, whereas ECoT relies solely on text input. We also illustrate an example of spatial reasoning.
• Figure 2: Construction of our hierarchical embodied dataset. We first segment the trajectory. Then, we generate the 3D spatial movement that requires to transition to the end state of the segment. Based on segments, we recognize the 2D gripper position and generate the grounded task reasoning.
• Figure 3: The overview of Emma-X fine-tuned from OpenVLA using our hierarchical embodiment dataset.
• Figure 4: Experimental results on different categories of real-world robot tasks.
• Figure 5: Qualitative examples of successful and failed cases with Emma-X on real-world robot testing.