ReFineVLA: Reasoning-Aware Teacher-Guided Transfer Fine-Tuning

TL;DR

ReFineVLA tackles the limited explicit reasoning in Vision-Language-Action robots by injecting multimodal chain-of-thought-style rationales generated by an expert teacher during fine-tuning. The framework employs selective transfer to preserve pretrained generalization while updating higher-level reasoning modules, optimizing a joint loss that combines action prediction with rationale generation. Empirical results across SimplerEnv WidowX/Google scenarios show consistent improvements over strong baselines and demonstrate more interpretable attention patterns and robust long-horizon reasoning. This work advances robust multimodal understanding in robot manipulation and offers a scalable path toward reasoning-enabled generalist VLA policies.

Abstract

Vision-Language-Action (VLA) models have gained much attention from the research community thanks to their strength in translating multimodal observations with linguistic instructions into robotic actions. Despite their recent advancements, VLAs often overlook the explicit reasoning and only learn the functional input-action mappings, omitting these crucial logical steps for interpretability and generalization for complex, long-horizon manipulation tasks. In this work, we propose \textit{ReFineVLA}, a multimodal reasoning-aware framework that fine-tunes VLAs with teacher-guided reasons. We first augment robotic datasets with reasoning rationales generated by an expert teacher model, guiding VLA models to learn to reason about their actions. Then, we use \textit{ReFineVLA} to fine-tune pre-trained VLAs with the reasoning-enriched datasets, while maintaining their inherent generalization abilities and boosting reasoning capabilities. In addition, we conduct an attention map visualization to analyze the alignment among visual attention, linguistic prompts, and to-be-executed actions of \textit{ReFineVLA}, showcasing its ability to focus on relevant tasks and actions. Through the latter step, we explore that \textit{ReFineVLA}-trained models exhibit a meaningful attention shift towards relevant objects, highlighting the enhanced multimodal understanding and improved generalization. Evaluated across manipulation tasks, \textit{ReFineVLA} outperforms the state-of-the-art baselines. Specifically, it achieves an average increase of $5.0\%$ success rate on SimplerEnv WidowX Robot tasks, improves by an average of $8.6\%$ in variant aggregation settings, and by $1.7\%$ in visual matching settings for SimplerEnv Google Robot tasks. The source code will be publicly available.

Figures (7)

• Figure 1: Attention Visualization: Attention maps of action tokens in standard VLAs, illustrating the narrow focus on visual cues.
• Figure 2: Multimodal Robotic Instruction Understanding with Chain-of-Thought Reasoning: An illustrative example depicts a single annotated data sample from a robotic manipulator grounded task planning. The task is for a robot to place a spoon into a cup, given an observation of a cluttered scene and natural language instructions. The input prompt guides the robot to reason through a sequence of structured questions: (1) Observation -- identifying objects in the image; (2) Situation Analysis -- understanding the context; (3) Spatial Reasoning -- analyzing object relationships; and (4) Task Planning -- formulating an action plan in logical steps. The annotated response includes step-by-step reasoning under each category, leading to a detailed plan of robot actions involving position, rotation, and gripper control for low-level motor commands.
• Figure 3: ReFineVLA's Training Flow: A fine-tuning framework that enhances VLA models with explicit multimodal reasoning, guided by rationales from a teacher model. These rationales cover visual cues, spatial reasoning, and task planning and are injected during training via action and reasoning losses. The learner integrates visual-linguistic inputs, infuses reasoning, and outputs interpretable actions for closed-loop control.
• Figure 4: Attention Visualization of ReFineVLA compared to SpatialVLA:RefineVLA shows better attention to related entities within the given observations conditioned by the input prompts than what SpatialVLA does.
• Figure 5: Chain-of-Thought Reasoning of ReFineVLA:RefineVLA shows step-by-step reasoning to accomplish the prompted task given the initial observation. The examples illustrate the queries (1) for placing the coke can near the orange positioned in the tabletop settings and (2) for closing the drawer while it is opening.