ActionFlow: A Pipelined Action Acceleration for Vision Language Models on Edge

TL;DR

ActionFlow tackles the latency bottleneck of Vision-Language-Action models on edge devices by introducing a system-level, training-free approach that reframes autoregressive inference as a cross-request macro-pipeline of micro-requests. By packing and overlapping memory-bound Decode phases of past requests with compute-bound Prefill of the current step, and by employing a Cross-Request State Packed Forward operator together with a Unified KV Ring Buffer, ActionFlow converts many small memory operations into a single dense computation. The method yields about a $2.55\times$ speedup on OpenVLA-7B across edge and high-end edge hardware without retraining and without modifying model accuracy, demonstrating practical real-time embodied AI on constrained platforms. This work enables robust, real-time dynamic manipulation in edge robotics and provides a reusable system-level design for accelerating autoregressive VLA workloads.

Abstract

Vision-Language-Action (VLA) models have emerged as a unified paradigm for robotic perception and control, enabling emergent generalization and long-horizon task execution. However, their deployment in dynamic, real-world environments is severely hin dered by high inference latency. While smooth robotic interaction requires control frequencies of 20 to 30 Hz, current VLA models typi cally operate at only 3-5 Hz on edge devices due to the memory bound nature of autoregressive decoding. Existing optimizations often require extensive retraining or compromise model accuracy. To bridge this gap, we introduce ActionFlow, a system-level inference framework tailored for resource-constrained edge plat forms. At the core of ActionFlow is a Cross-Request Pipelin ing strategy, a novel scheduler that redefines VLA inference as a macro-pipeline of micro-requests. The strategy intelligently batches memory-bound Decode phases with compute-bound Prefill phases across continuous time steps to maximize hardware utilization. Furthermore, to support this scheduling, we propose a Cross Request State Packed Forward operator and a Unified KV Ring Buffer, which fuse fragmented memory operations into efficient dense computations. Experimental results demonstrate that ActionFlow achieves a 2.55x improvement in FPS on the OpenVLA-7B model without retraining, enabling real-time dy namic manipulation on edge hardware. Our work is available at https://anonymous.4open.science/r/ActionFlow-1D47.

Figures (5)

• Figure 1: The process of Vision Language Model/Action
• Figure 2: Deconstructing the performance bottleneck. (a) Hierarchical latency breakdown showing VLA inference process (b) Roofline analysis for Jetson AGX Orin platform
• Figure 3: Overview of ActionFlow pipeline
• Figure 4: Workflow of the Packed Layer with Unified KV Ring Buffer. The blue blocks represent the KV cache corresponding to the current input frame.
• Figure 5: Sensitivity analysis of throughput (FPS) on AGX Orin (top) and RTX 5090 (down) across varying Action Lengths ($K$) and Prefill Lengths ($L_P$).