Do What You Say: Steering Vision-Language-Action Models via Runtime Reasoning-Action Alignment Verification

TL;DR

Do What You Say presents SEAL, a training-free runtime steering framework that enforces embodied CoT faithfulness in Vision-Language-Action models by verifying that the outcomes of sampled action sequences align with the model's own textual reasoning. The method combines a reasoning VLA trained on Gemini-annotated data with a Hypothesize-Predict-Verify loop that uses a Vision-Language Model verifier to select the plan-aligned sequence, enabling robust long-horizon and compositional behavior without retraining. Empirical results on LIBERO-based tasks show up to 15% gains, improved robustness to semantic and visual out-of-distribution shifts, and scalable performance as data and compute increase. This approach demonstrates that runtime verification of reasoning-to-action alignment can significantly enhance reliability and generalization in embodied AI systems.

Abstract

Reasoning Vision Language Action (VLA) models improve robotic instruction-following by generating step-by-step textual plans before low-level actions, an approach inspired by Chain-of-Thought (CoT) reasoning in language models. Yet even with a correct textual plan, the generated actions can still miss the intended outcomes in the plan, especially in out-of-distribution (OOD) scenarios. We formalize this phenomenon as a lack of embodied CoT faithfulness, and introduce a training-free, runtime policy steering method for reasoning-action alignment. Given a reasoning VLA's intermediate textual plan, our framework samples multiple candidate action sequences from the same model, predicts their outcomes via simulation, and uses a pre-trained Vision-Language Model (VLM) to select the sequence whose outcome best aligns with the VLA's own textual plan. Only executing action sequences that align with the textual reasoning turns our base VLA's natural action diversity from a source of error into a strength, boosting robustness to semantic and visual OOD perturbations and enabling novel behavior composition without costly re-training. We also contribute a reasoning-annotated extension of LIBERO-100, environment variations tailored for OOD evaluation, and demonstrate up to 15% performance gain over prior work on behavior composition tasks and scales with compute and data diversity. Project Website at: https://yilin-wu98.github.io/steering-reasoning-vla/

Figures (9)

• Figure 1: Method Overview. Reasoning Vision Language Action (VLA) models interleave textual planning and action generation. After generating a text plan which describes intermediate goals, we sample a batch of action sequences, forward simulate their outcomes until the model switches to think again. We then use a Vision Language model (VLM) verifier to score alignment between the action's outcomes and the text plan. This improves the embodied CoT faithfulness at runtime by executing only action samples that achieve the outcome of the text plan.
• Figure 2: Dataset Scaling Results. Our method, SEAL, consistently outperforms all baselines by enforcing reasoning-action alignment with increased performance as the training data scales, especially on challenging behavior composition tasks.
• Figure 3: Comparison Between Verification Methods. (left) For ID tasks, both the Q-value based verifier in $\bm{\pi_0}$-V-GPS and our VLM verifier can accurately predict task progress. (right) For behavior composition tasks requiring stronger generalization capabilities, the Q-value overestimates task progress, failing to provide useful signal to guide $\bm{\pi_0}$. However, our VLM verifier preserves the commonsense reasoning and correctly guides the base $\bm{\pi_0}$-reason policy towards the task goal.
• Figure 4: Robustness to Semantic and Visual OOD Shifts. We evaluate four different OOD variations for each task in Libero-10. The performance is averaged across 10 tasks in the task suite with 50 trials per task. SEAL maintains the best performance across all OOD shifts, even in the most challenging Visual-Viewpoint OOD scenario.
• Figure 5: OOD Shifts & Behavior Composition Examples. We visualize SEAL and $\bm{\pi_0}$-reason. We mark a successful rollout with a green box and a failure with red. SEAL (on the left) can reliably generate successful rollouts to complete the tasks. $\bm{\pi_0}$-reason policy (on the right) fails due to imprecision or by completing a totally different textual plan.