Unified Embodied VLM Reasoning with Robotic Action via Autoregressive Discretized Pre-training

TL;DR

The paper tackles the problem of achieving both broad generalization and high-precision action in open-world robotic manipulation, addressing a gap where Vision-Language-Action systems struggle to jointly reason and execute with precision. It introduces GenieReasoner, a unified autoregressive framework that grounds high-level reasoning in discrete action tokens through FACT, a flow-matching action tokenizer, enabling end-to-end optimization in a single space. The authors propose ERIQ, a large-scale embodied reasoning benchmark with 6,052 QA pairs across four pillars, to decouple reasoning from execution and reveal a strong link between embodied reasoning and end-to-end manipulation success. Empirical results show GenieReasoner outperforms continuous-action and discrete-action baselines on ERIQ and in real-world deployments, with FACT achieving high reconstruction fidelity at compact token lengths, demonstrating a principled approach to bridging semantic planning and precise motor control for robust, general-purpose robotic manipulation.

Abstract

General-purpose robotic systems operating in open-world environments must achieve both broad generalization and high-precision action execution, a combination that remains challenging for existing Vision-Language-Action (VLA) models. While large Vision-Language Models (VLMs) improve semantic generalization, insufficient embodied reasoning leads to brittle behavior, and conversely, strong reasoning alone is inadequate without precise control. To provide a decoupled and quantitative assessment of this bottleneck, we introduce Embodied Reasoning Intelligence Quotient (ERIQ), a large-scale embodied reasoning benchmark in robotic manipulation, comprising 6K+ question-answer pairs across four reasoning dimensions. By decoupling reasoning from execution, ERIQ enables systematic evaluation and reveals a strong positive correlation between embodied reasoning capability and end-to-end VLA generalization. To bridge the gap from reasoning to precise execution, we propose FACT, a flow-matching-based action tokenizer that converts continuous control into discrete sequences while preserving high-fidelity trajectory reconstruction. The resulting GenieReasoner jointly optimizes reasoning and action in a unified space, outperforming both continuous-action and prior discrete-action baselines in real-world tasks. Together, ERIQ and FACT provide a principled framework for diagnosing and overcoming the reasoning-precision trade-off, advancing robust, general-purpose robotic manipulation.

Figures (11)

• Figure 1: We introduce the GenieReasoner system. (Left) Our system leverages large-scale general and embodied multimodal data to co-optimize high-level reasoning and low-level control within a unified autoregressive transformer. (Center) To bridge the gap between discrete planning and continuous execution, we introduce FACT, a novel action tokenizer that utilizes flow matching to reconstruct high-fidelity trajectories from quantized tokens. (Right) This unified design yields state-of-the-art results: GenieReasoner achieves a 41% accuracy improvement on our proposed ERIQ for embodied reasoning and demonstrates significantly lower reconstruction error (MSE) compared to $\pi_0$-FAST. Consequently, our model outperforms flow-based baselines (e.g., $\pi_{0.5}$) in real-world robot manipulation tasks.
• Figure 2: Illustration of the ERIQ benchmark. Example samples from the four major categories of embodied reasoning.
• Figure 3: Distribution of the ERIQ benchmark across its 15 fine-grained sub-tasks.
• Figure 4: The GenieReasoner system architecture. (a) Training: Our unified pipeline jointly optimizes the VLM backbone for multimodal reasoning and robotic control by tokenizing continuous actions into a discrete latent space. This process encompasses the second and third stages of the training recipe (\ref{['sec:implementation']}), where General VQA data is incorporated during the second stage to preserve foundational vision-language knowledge. (b) Inference: Discrete action codes generated by the VLM backbone are decoded into continuous control signals via the FACT decoder, ensuring high-precision manipulation that is semantically grounded in the task instructions.
• Figure 5: The FACT Action Tokenizer. We discretize continuous robot actions into compact tokens via a VQ-encoder, enabling autoregressive modeling by the VLM. To preserve control precision, the decoder utilizes flow matching to reconstruct smooth, continuous trajectories from the quantized tokens and Gaussian noise z.