Asynchronous Fast-Slow Vision-Language-Action Policies for Whole-Body Robotic Manipulation

TL;DR

The paper tackles the bottleneck of applying large vision–language models to real-time, whole-body robotic manipulation by decoupling semantic reasoning from high-frequency control through a fully asynchronous fast–slow VLA framework, DuoCore-FS. It introduces a bridge buffer to carry latent semantic and reasoning representations from a slow VLM-driven pathway to a fast diffusion-policy controller, and an action tokenizer based on RVQ-VAE to compactly represent whole-body motions. The training employs a two-stage, cross-timescale scheme to align slow and fast components, and experiments on the Astribot S1 show improved responsiveness and task success over synchronous baselines, with roughly threefold faster action-generation at comparable model sizes. The approach enables large VLMs to inform fast, robust manipulation with real-time performance, advancing practical deployment of VLA policies in dynamic, real-world scenarios.

Abstract

Most Vision-Language-Action (VLA) systems integrate a Vision-Language Model (VLM) for semantic reasoning with an action expert generating continuous action signals, yet both typically run at a single unified frequency. As a result, policy performance is constrained by the low inference speed of large VLMs. This mandatory synchronous execution severely limits control stability and real-time performance in whole-body robotic manipulation, which involves more joints, larger motion spaces, and dynamically changing views. We introduce a truly asynchronous Fast-Slow VLA framework (DuoCore-FS), organizing the system into a fast pathway for high-frequency action generation and a slow pathway for rich VLM reasoning. The system is characterized by two key features. First, a latent representation buffer bridges the slow and fast systems. It stores instruction semantics and action-reasoning representation aligned with the scene-instruction context, providing high-level guidance to the fast pathway. Second, a whole-body action tokenizer provides a compact, unified representation of whole-body actions. Importantly, the VLM and action expert are still jointly trained end-to-end, preserving unified policy learning while enabling asynchronous execution. DuoCore-FS supports a 3B-parameter VLM while achieving 30 Hz whole-body action-chunk generation, approximately three times as fast as prior VLA models with comparable model sizes. Real-world whole-body manipulation experiments demonstrate improved task success rates and significantly enhanced responsiveness compared to synchronous Fast-Slow VLA baselines. The implementation of DuoCore-FS, including training, inference, and deployment, is provided to commercial users by Astribot as part of the Astribot robotic platform.

Figures (6)

• Figure 1: Overview of the proposed fast--slow asynchronous VLA policy framework. The slow system (bottom) operates at a low frequency of 1--3 Hz, where a large Vision-Language Model (VLM) parses task instructions, visual observations, and proprioceptive states to produce high-level semantic hidden states, including text embeddings, reasoning features, and learnable fusion queries. These representations are periodically written into a bridge buffer. The fast system (top) fetches the latest latent semantic representations from the bridge buffer at a high control rate of 25--30 Hz, integrates them with current visual features and proprioceptive states, and uses a Transformer-based diffusion-policy decoder to generate smooth, continuous, and fully coordinated whole-body actions.
• Figure 2: Cross-timescale co-training. The slow system processes observation $o_{t_0}$, while the fast system receives a shifted observation $o_{t_0+\Delta}$ with $\Delta \sim \mathcal{U}[0,\,\Delta_{\max}]$. This sampling strategy mimics the asynchronous timing during real deployment, enabling consistent end-to-end optimization.
• Figure 3: Popcorn-scooping task pipeline.
• Figure 4: In-distribution benchmark setup. The cup and the scoop are placed at diverse locations. The figure provides a schematic overview and does not enumerate all test positions.
• Figure 5: Illustration of anomaly cases