Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

StreamVLA: Breaking the Reason-Act Cycle via Completion-State Gating

Hang Wu, Tongqing Chen, Jiasen Wang, Xiaotao Li, Lu Fang

TL;DR

StreamVLA tackles the latency-accuracy trade-off in long-horizon robotic manipulation by unifying slow planning and fast control within a single backbone and introducing a future-completion imagination head plus a lock-and-gated mechanism. The model gates heavy autoregressive reasoning to sub-task transitions, using a visually imagined completion state as a stable goal anchor to guide low-latency control and maintain task coherence. Empirical results show state-of-the-art performance on LIBERO (98.5% success) with a 48% latency reduction, strong robustness to dynamic interference, and real-world success across challenging tasks. The approach offers a scalable path for deploying large multimodal models in real-time robotic systems and points to future work integrating tactile feedback and other modalities.

Abstract

Long-horizon robotic manipulation requires bridging the gap between high-level planning (System 2) and low-level control (System 1). Current Vision-Language-Action (VLA) models often entangle these processes, performing redundant multimodal reasoning at every timestep, which leads to high latency and goal instability. To address this, we present StreamVLA, a dual-system architecture that unifies textual task decomposition, visual goal imagination, and continuous action generation within a single parameter-efficient backbone. We introduce a "Lock-and-Gated" mechanism to intelligently modulate computation: only when a sub-task transition is detected, the model triggers slow thinking to generate a textual instruction and imagines the specific visual completion state, rather than generic future frames. Crucially, this completion state serves as a time-invariant goal anchor, making the policy robust to execution speed variations. During steady execution, these high-level intents are locked to condition a Flow Matching action head, allowing the model to bypass expensive autoregressive decoding for 72% of timesteps. This hierarchical abstraction ensures sub-goal focus while significantly reducing inference latency. Extensive evaluations demonstrate that StreamVLA achieves state-of-the-art performance, with a 98.5% success rate on the LIBERO benchmark and robust recovery in real-world interference scenarios, achieving a 48% reduction in latency compared to full-reasoning baselines.

StreamVLA: Breaking the Reason-Act Cycle via Completion-State Gating

TL;DR

StreamVLA tackles the latency-accuracy trade-off in long-horizon robotic manipulation by unifying slow planning and fast control within a single backbone and introducing a future-completion imagination head plus a lock-and-gated mechanism. The model gates heavy autoregressive reasoning to sub-task transitions, using a visually imagined completion state as a stable goal anchor to guide low-latency control and maintain task coherence. Empirical results show state-of-the-art performance on LIBERO (98.5% success) with a 48% latency reduction, strong robustness to dynamic interference, and real-world success across challenging tasks. The approach offers a scalable path for deploying large multimodal models in real-time robotic systems and points to future work integrating tactile feedback and other modalities.

Abstract

Long-horizon robotic manipulation requires bridging the gap between high-level planning (System 2) and low-level control (System 1). Current Vision-Language-Action (VLA) models often entangle these processes, performing redundant multimodal reasoning at every timestep, which leads to high latency and goal instability. To address this, we present StreamVLA, a dual-system architecture that unifies textual task decomposition, visual goal imagination, and continuous action generation within a single parameter-efficient backbone. We introduce a "Lock-and-Gated" mechanism to intelligently modulate computation: only when a sub-task transition is detected, the model triggers slow thinking to generate a textual instruction and imagines the specific visual completion state, rather than generic future frames. Crucially, this completion state serves as a time-invariant goal anchor, making the policy robust to execution speed variations. During steady execution, these high-level intents are locked to condition a Flow Matching action head, allowing the model to bypass expensive autoregressive decoding for 72% of timesteps. This hierarchical abstraction ensures sub-goal focus while significantly reducing inference latency. Extensive evaluations demonstrate that StreamVLA achieves state-of-the-art performance, with a 98.5% success rate on the LIBERO benchmark and robust recovery in real-world interference scenarios, achieving a 48% reduction in latency compared to full-reasoning baselines.
Paper Structure (21 sections, 8 equations, 5 figures, 4 tables)

This paper contains 21 sections, 8 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Vision-Language-Action Foundations
  4. Long-Horizon Planning and Visual Foresight
  5. Efficient and Streaming Multimodal Reasoning
  6. Method
  7. Overview
  8. Future-Completion Visual Imagination
  9. Lock-and-Gated Mechanism
  10. Unified Multi-Modal Output
  11. Data Annotation
  12. Training Strategy
  13. Experiments
  14. Experimental Setup
  15. Main Results
  16. ...and 6 more sections

Figures (5)

  • Figure 1: StreamVLA Overview. A dual-system architecture with Slow Thinking (purple path) for adaptive sub-task planning and future imagination, and Fast Action (blue path) for continuous control. By using predicted future images to gate reasoning, it skips redundant computation when the sub-task is incomplete (72% of the time), achieving a 48% reduction in average latency (244ms $\to$ 128ms).
  • Figure 2: The StreamVLA Architecture. Our framework unifies sparse reasoning with high-frequency control through a Lock-and-Gated mechanism. The shared VLA backbone processes multi-view observations. A lightweight Gating Module continuously compares the current state against the locked Completion Image. If the discrepancy is low (sub-task ongoing), the system operates in Skip Mode, bypassing the computationally expensive heads (System 2) and reusing the cached sub-task plan. When a transition is detected, Full Mode is triggered to generate a new text plan and visual completion goal. The Action Expert (System 1) synthesizes precise motor trajectories via Flow Matching, conditioned on the locked high-level semantic and visual intents from System 2.
  • Figure 3: Future-Completion Visual Imagination Examples. Three LIBERO tasks showing (left to right): current observation, predicted future completion frame, and ground truth completion state. Unlike methods that predict the next frame ($t+\Delta t$), StreamVLA generates the sub-task completion state (Goal). This goal remains stable throughout the execution of the sub-task, serving as a robust anchor for the gating mechanism.
  • Figure 4: Lock-and-Gated Mechanism.(a) The Gating Module computes a Discrepancy Score $d_t$ by comparing current observation against the locked goal. (b) Control Logic: If $d_t > \tau$ (Gap Large), the system locks reasoning and executes actions (Skip Mode). If $d_t \leq \tau$ (Gap Closed), it triggers re-planning (Full Mode).
  • Figure 5: Future-completion prediction examples. (a) RoboTwin 2.0 bimanual hammer task. (b) Two real-world Piper spell task. Each row shows current observation, predicted and ground future-completion frame.