MoTVLA: A Vision-Language-Action Model with Unified Fast-Slow Reasoning

TL;DR

MoTVLA tackles the latency–steerability trade-off in vision-language–driven robotic policies by integrating fast and slow reasoning in a single Mixture-of-Transformers framework. A pre-trained generalist handles slow, general reasoning, while a domain expert provides fast, motion-decomposition reasoning, both sharing knowledge through a unified global attention; an action expert diffusion policy translates reasoning into language-conditioned motor actions. Training employs a two-stage curriculum with domain-expert supervised fine-tuning and diffusion-based control, leveraging large-scale VLM and robotics datasets to preserve general intelligence while acquiring domain-specific skills. Experiments across NLP benchmarks, ManiSkill simulation, and real-world robotics demonstrate MoTVLA’s faster inference, improved reasoning quality, and robust language steerability for diverse manipulation tasks. The work suggests a scalable path toward open-world, language-guided robotic systems by fusing high-level reasoning with low-level control in a unified VLA model.

Abstract

Integrating visual-language instructions into visuomotor policies is gaining momentum in robot learning for enhancing open-world generalization. Despite promising advances, existing approaches face two challenges: limited language steerability when no generated reasoning is used as a condition, or significant inference latency when reasoning is incorporated. In this work, we introduce MoTVLA, a mixture-of-transformers (MoT)-based vision-language-action (VLA) model that integrates fast-slow unified reasoning with behavior policy learning. MoTVLA preserves the general intelligence of pre-trained VLMs (serving as the generalist) for tasks such as perception, scene understanding, and semantic planning, while incorporating a domain expert, a second transformer that shares knowledge with the pretrained VLM, to generate domain-specific fast reasoning (e.g., robot motion decomposition), thereby improving policy execution efficiency. By conditioning the action expert on decomposed motion instructions, MoTVLA can learn diverse behaviors and substantially improve language steerability. Extensive evaluations across natural language processing benchmarks, robotic simulation environments, and real-world experiments confirm the superiority of MoTVLA in both fast-slow reasoning and manipulation task performance.

Figures (14)

• Figure 1: Unified fast–slow reasoning in MoTVLA and its reasoning performance. MoTVLA unifies fast–slow reasoning through a Mixture-of-Transformers architecture, in which the input modalities are first decomposed, followed by knowledge sharing, and finally decomposed again from a functional perspective. As a result, MoTVLA not only learns both general and domain-specific knowledge effectively, but also achieves superior reasoning efficiency.
• Figure 2: The universal framework of MoTVLA. MoTVLA adopts a Mixture-of-Transformers architecture comprising a generalist, a domain expert, and an action expert. Its reasoning backbone follows a decomposition–composition–decomposition pipeline: multimodal inputs are first processed independently, then integrated through a unified global self-attention mechanism, and finally decoupled at the output to perform slow and fast reasoning via the generalist and domain expert, respectively. The fast reasoning module decomposes robotic motions, and the resulting motion representation, together with visual and physical states, condition the action expert. This design ensures that the learned policy aligns with motion instructions and enhances language steerability, even under ambiguous prompts.
• Figure 3: A concrete example of fast and slow reasoning during inference. When asked a general question, the generalist performs slow reasoning and provides a detailed verbal response. When a task-execution prompt is issued, the domain expert performs fast reasoning to generate decomposed intermediate motions for action execution.
• Figure 4: Fast reasoning instances and slow reasoning benchmark.$*$ indicates that the generalist is fine-tuned on the data originally intended for training the domain expert.
• Figure 5: Evaluation for manipulation tasks. MoTVLA is rigorously evaluated in both simulation and real-world experiments. The testing suite encompasses diverse case types and variations, including ambiguous instruction prompts that require strong reasoning capability and language-steered behavioral policies.