Hybrid Training for Vision-Language-Action Models

TL;DR

This work addresses the trade-off between leveraging reasoning traces (CoT) for improved Vision-Language-Action model performance and maintaining fast action inference in robotics. It introduces Hybrid Training (HyT), a modality-conditioned objective that trains a single VLA to output Action, Think, or Follow distributions, while enabling fast act-only inference at test time. HyT achieves performance on par with or surpassing Embodied CoT and related hierarchies on ClevrSkills and LIBERO, and demonstrates real-world gains on a 320-trajectory robot dataset, with notably better handling of out-of-distribution scenarios. The approach offers practical impact by delivering flexible, interpretable, and efficient robotic policies that can adapt to varying inference needs without incurring substantial latency penalties.

Abstract

Using Large Language Models to produce intermediate thoughts, a.k.a. Chain-of-thought (CoT), before providing an answer has been a successful recipe for solving complex language tasks. In robotics, similar embodied CoT strategies, generating thoughts before actions, have also been shown to lead to improved performance when using Vision-Language-Action models (VLAs). As these techniques increase the length of the model's generated outputs to include the thoughts, the inference time is negatively affected. Delaying an agent's actions in real-world executions, as in robotic manipulation settings, strongly affects the usability of a method, as tasks require long sequences of actions. However, is the generation of long chains-of-thought a strong prerequisite for achieving performance improvements? In this work, we explore the idea of Hybrid Training (HyT), a framework that enables VLAs to learn from thoughts and benefit from the associated performance gains, while enabling the possibility to leave out CoT generation during inference. Furthermore, by learning to conditionally predict a diverse set of outputs, HyT supports flexibility at inference time, enabling the model to either predict actions directly, generate thoughts or follow instructions. We evaluate the proposed method in a series of simulated benchmarks and real-world experiments.

Figures (8)

• Figure 1: Hybrid Training (HyT) of VLAs increases the agent's performance similarly to ECoT, but also maintains the same fast inference as standard VLAs. Performance refers to the ClevrSkills experiments (9 tasks, 3000 demos) in the Experiments section.
• Figure 2: Hybrid Training (HyT) framework. Given a set of inputs, on the left, including a modality variable, the VLA model learns to conditionally generate a variety of outputs. Examples for the 'think' and 'act' conditional distributions are presented on the right.
• Figure 3: ClevrSkills benchmark Aggregated performance on 9 ClevrSkills environments (examples shown on the left). Shaded areas indicate standard errors.
• Figure 4: Instruction-following and other inference modalities. Comparing performance when using own generated thoughts (default) or following instructions generated by an oracle.
• Figure 5: LIBERO benchmark Aggregated performance on 4 LIBERO task suites (examples shown on the left). Mean performance evaluated on 100 evaluation episodes.

Theorems & Definitions (1)

• Definition 4.1: Hybrid Training