VOTE: Vision-Language-Action Optimization with Trajectory Ensemble Voting

TL;DR

VLA models for robotic manipulation suffer from heavy action-token generation and underutilization of generated actions, limiting edge deployment. The authors propose VOTE, a tokenizer-free training framework that compresses action output to a single $<$ACT$>$ token and employs a bottleneck MLP head to generate an action chunk in parallel, reducing both training and inference costs. For inference, an ensemble voting mechanism combines current and historical predictions via cosine similarity to improve action reliability. Across LIBERO and SimplerEnv benchmarks, VOTE delivers state-of-the-art success rates and large speedups (e.g., up to $46$ Hz on edge devices with substantial throughput gains), demonstrating practical deployability and robustness. These results show that reducing output tokens and intelligent ensemble strategies can significantly improve both efficiency and performance of VLA-based robotic policies.

Abstract

Recent large-scale Vision Language Action (VLA) models have shown superior performance in robotic manipulation tasks guided by natural language. However, current VLA models suffer from two drawbacks: (i) generation of massive tokens leading to high inference latency and increased training cost, and (ii) insufficient utilization of generated actions resulting in potential performance loss. To address these issues, we develop a training framework to finetune VLA models for generating significantly fewer action tokens with high parallelism, effectively reducing inference latency and training cost. Furthermore, we introduce an inference optimization technique with a novel voting-based ensemble strategy to combine current and previous action predictions, improving the utilization of generated actions and overall performance. Our results demonstrate that we achieve superior performance compared with state-of-the-art VLA models, achieving significantly higher success rates and 39$\times$ faster inference than OpenVLA with 46 Hz throughput on edge platforms, demonstrating practical deployability. The code is available at https://github.com/LukeLIN-web/VOTE.

Figures (5)

• Figure 1: The whole VOTE pipeline, where we generate the next following $n$ actions in parallel and adopt the ensemble voting strategy for the accurate current action prediction.
• Figure 2: Latency for SpatialVLA, CogACT, and OpenVLA.
• Figure 3: Vote Action Ensemble. Illustration of our action ensemble strategy with K = 3 (using the last 3 historical action predictions) as an example. Historical predictions and the current prediction form a voting committee to jointly determine the final action to execute. For example, when t = 3, more than half of the candidate actions differ from the current prediction, voting not-similar. Therefore, we discard the current prediction and instead compute the final ensembled action by averaging the previous 3 historical action predictions which vote not-similar.
• Figure 4: Ablation on hyperparameter $\tau$. WR denotes WidowX Robot in SimplerEnv setting. $\tau$ is a hyperparameter specific to Our Method and that the baselines are shown for comparison and do not depend on $\tau$
• Figure 5: Training Loss Across LIBERO Datasets