FASTer: Toward Efficient Autoregressive Vision Language Action Modeling via Neural Action Tokenization

TL;DR

FASTer tackles the bottleneck of action tokenization in autoregressive Vision-Language-Action models by introducing FASTerVQ, a high‑compression, high‑fidelity action tokenizer, and FASTerVLA, an efficient BAR-enabled autoregressive policy with a lightweight action expert. The approach achieves near-lossless action reconstruction with significantly fewer tokens and demonstrates strong generalization across embodiments and backbones. Empirical results show state-of-the-art in-distribution performance and robust out-of-distribution transfer, with substantial reductions in inference latency compared to prior AR VLA methods. Overall, FASTer provides a scalable, transferable framework for efficient multimodal robotic control that can leverage pretrained VLM backbones without retraining.

Abstract

Autoregressive vision-language-action (VLA) models have recently demonstrated strong capabilities in robotic manipulation. However, their core process of action tokenization often involves a trade-off between reconstruction fidelity and inference efficiency. We introduce FASTer, a unified framework for efficient and generalizable robot learning that integrates a learnable tokenizer with an autoregressive policy built upon it. FASTerVQ encodes action chunks as single-channel images, capturing global spatio-temporal dependencies while maintaining a high compression ratio. FASTerVLA builds on this tokenizer with block-wise autoregressive decoding and a lightweight action expert, achieving both faster inference and higher task performance. Extensive experiments across simulated and real-world benchmarks show that FASTerVQ delivers superior reconstruction quality, high token utilization, and strong cross-task and cross-embodiment generalization, while FASTerVLA further improves overall capability, surpassing previous state-of-the-art VLA models in both inference speed and task performance.

Figures (17)

• Figure 1: FASTer combines a learnable action tokenizer (FASTerVQ) and an autoregressive VLA model (FASTerVLA), achieving efficient compression, fast control, and strong performance across eight real and simulated embodiments.
• Figure 1: Policy performance on Libero and Simpler-Bridge benchmarks.
• Figure 2: FASTerVQ. (a) The raw action sequence is patchified into compact tokens based on their embodied configuration. (b) FASTerVQ applies DCT and L1 reconstruction losses and adopts RVQ, encoding actions into $N_c$ code levels; each level can be reshaped into a $C_h \times C_a$ tensor.
• Figure 3: FASTerVLA.(a) The model takes RGB images, proprioceptive states, and a language instruction as inputs to a transformer–based VLM. An Action Expert autoregressively generates discrete action tokens, which a VQ decoder maps into the final continuous action sequence. (b) Codes are generated codebook–wise before advancing along the temporal horizon, yielding greater stability than horizon–first decoding (red arrows denote the decoding order). (c) Left: vanilla causal mask; Right: Block-wise causal mask, where tokens within each block attend to preceding and intra-block tokens. Token colors denote different modalities.
• Figure 4: Policy performance across embodiments and environments. Results are reported in the in-distribution setting, covering two real-world embodiments and three simulated setups.