AC^2-VLA: Action-Context-Aware Adaptive Computation in Vision-Language-Action Models for Efficient Robotic Manipulation

TL;DR

The paper addresses the latency and compute bottleneck of closed-loop Vision-Language-Action models in robotics by introducing AC^2-VLA, a framework that conditions computation on action context. A lightweight action-prior router unifies cognition caching, token pruning, and conditional layer skipping, guided by the previous action state and multimodal observations, and trained with action-guided self-distillation to preserve dense-policy behavior. Empirical results on SIMPLER benchmarks show up to a 1.79× speedup with only 29.4% of FLOPs of the dense baseline and with comparable or improved task success, along with extensive ablations demonstrating complementary gains from each efficiency axis. The work highlights that aligning computation with action context can achieve substantial efficiency gains without sacrificing robustness, pointing to adaptive inference as a key ingredient for scalable generalist robot policies.

Abstract

Vision-Language-Action (VLA) models have demonstrated strong performance in robotic manipulation, yet their closed-loop deployment is hindered by the high latency and compute cost of repeatedly running large vision-language backbones at every timestep. We observe that VLA inference exhibits structured redundancies across temporal, spatial, and depth dimensions, and that most existing efficiency methods ignore action context, despite its central role in embodied tasks. To address this gap, we propose Action-Context-aware Adaptive Computation for VLA models (AC^2-VLA), a unified framework that conditions computation on current visual observations, language instructions, and previous action states. Based on this action-centric context, AC^2-VLA adaptively performs cognition reuse across timesteps, token pruning, and selective execution of model components within a unified mechanism. To train the adaptive policy, we introduce an action-guided self-distillation scheme that preserves the behavior of the dense VLA policy while enabling structured sparsification that transfers across tasks and settings. Extensive experiments on robotic manipulation benchmarks show that AC^2-VLA achieves up to a 1.79\times speedup while reducing FLOPs to 29.4% of the dense baseline, with comparable task success.

Figures (5)

• Figure 1: Comparison of efficient VLA computation strategies. Existing methods typically apply cache reuse, token pruning, or layer skipping based on visual or heuristic cues in an uncoordinated manner, resulting in action-context-agnostic efficiency. In contrast, AC$^2$-VLA leverages action context to jointly gate cache reuse, token pruning, and layer skipping for action-context-aware efficiency.
• Figure 2: Overview of the proposed AC$^2$-VLA. At each timestep, the model builds an action-prior condition $\mathbf{c}_t$ from the current observation, instruction, and action context, and uses a unified router to generate token pruning, layer skipping, and cache reuse gates, enabling efficient computation and low-latency control.
• Figure 3: Adaptive layer execution and cache reuse over time.
• Figure 4: Left: input observation. Right: visualization of token-level importance predicted by the action-conditioned router, highlighting regions relevant to the current manipulation stage while suppressing distractors. The highlighted regions adapt with the action context, focusing computation on interaction-critical areas.
• Figure 5: Pareto frontier for token pruning and layer skipping on the SIMPLER benchmark.