ATA: Bridging Implicit Reasoning with Attention-Guided and Action-Guided Inference for Vision-Language Action Models

TL;DR

ATA is a novel training-free framework that introduces implicit reasoning into VLA inference through complementary attention-guided and action-guided strategies, thereby adaptively refining visual inputs without requiring extra training or annotations.

Abstract

Vision-Language-Action (VLA) models rely on current observations, including images, language instructions, and robot states, to predict actions and complete tasks. While accurate visual perception is crucial for precise action prediction and execution, recent work has attempted to further improve performance by introducing explicit reasoning during inference. However, such approaches face significant limitations. They often depend on data-intensive resources such as Chain-of-Thought (CoT) style annotations to decompose tasks into step-by-step reasoning, and in many cases require additional visual grounding annotations (e.g., bounding boxes or masks) to highlight relevant image regions. Moreover, they involve time-consuming dataset construction, labeling, and retraining, which ultimately results in longer inference sequences and reduced efficiency. To address these challenges, we propose ATA, a novel training-free framework that introduces implicit reasoning into VLA inference through complementary attention-guided and action-guided strategies. Unlike CoT or explicit visual-grounding methods, ATA formulates reasoning implicitly by integrating attention maps with an action-based region of interest (RoI), thereby adaptively refining visual inputs without requiring extra training or annotations. ATA is a plug-and-play implicit reasoning approach for VLA models, lightweight yet effective. Extensive experiments show that it consistently improves task success and robustness while preserving, and even enhancing, inference efficiency.

Figures (2)

• Figure 1: Overview of the proposed ATA framework. (Left) Given a language instruction and multimodal observations, the VLA model processes tokens through stacked layers to produce an action chunk. Our framework injects implicit reasoning cues via two complementary strategies: (i) Attention-guided reasoning extracts attention maps from intermediate layers to focus on task-relevant visual regions; (ii) Action-guided reasoning leverages the end-effector pose and camera parameters to construct a directional region of interest (RoI). (Right) In the original pipeline (top), an error at an early step may propagate along the horizon (up to $n$ steps), leading to task failure (marked with $\times$) and risky behaviors such as knocking over a bottle (highlighted with a red circle). The proposed pipeline (bottom) applies guidance at selected steps, resulting in a shorter effective horizon ($m<n$), successful task completion (marked with $\checkmark$), and more robust execution.
• Figure 2: Illustration of the action-guided strategy. The end-effector (EEF) pose $(\mathbf{R}^{W}_{\text{EEF}},\mathbf{t}^{W}_{\text{EEF}})$ is obtained in the world coordinate system and transformed into the camera coordinate $(\mathbf{R}^{C}_{\text{EEF}},\mathbf{t}^{C}_{\text{EEF}})$, allowing projection of the EEF direction onto the image plane. The resulting region of interest (RoI) is represented as a conic sector in the pixel coordinate system. For visualization, we overlay a semi-transparent red mask so that pixels inside the RoI appear in red, while the actual inference mask is computed from Eq. \ref{['eq:M_act']}.