ActiveVLA: Injecting Active Perception into Vision-Language-Action Models for Precise 3D Robotic Manipulation

TL;DR

ActiveVLA introduces active perception into Vision-Language-Action models by embedding a coarse-to-fine perceptual loop that first localizes 3D crucial regions from multi-view projections and then actively selects viewpoints and applies a 3D zoom-in to refine observations. The two-stage pipeline—3D crucial-area perception followed by 3D active perception—enables adaptive, high-resolution sensing for precise 3D manipulation. Across RLBench, COLOSSEUM, GemBench, and real-world tasks, ActiveVLA achieves state-of-the-art performance and strong generalization, demonstrating the practical benefits of hypothesis-driven viewpoint control in complex environments. This work highlights the significance of integrating active perception with large-scale vision-language backbones to improve robustness and precision in long-horizon robotic manipulation tasks.

Abstract

Recent advances in robot manipulation have leveraged pre-trained vision-language models (VLMs) and explored integrating 3D spatial signals into these models for effective action prediction, giving rise to the promising vision-language-action (VLA) paradigm. However, most existing approaches overlook the importance of active perception: they typically rely on static, wrist-mounted cameras that provide an end-effector-centric viewpoint. As a result, these models are unable to adaptively select optimal viewpoints or resolutions during task execution, which significantly limits their performance in long-horizon tasks and fine-grained manipulation scenarios. To address these limitations, we propose ActiveVLA, a novel vision-language-action framework that empowers robots with active perception capabilities for high-precision, fine-grained manipulation. ActiveVLA adopts a coarse-to-fine paradigm, dividing the process into two stages: (1) Critical region localization. ActiveVLA projects 3D inputs onto multi-view 2D projections, identifies critical 3D regions, and supports dynamic spatial awareness. (2) Active perception optimization. Drawing on the localized critical regions, ActiveVLA uses an active view selection strategy to choose optimal viewpoints. These viewpoints aim to maximize amodal relevance and diversity while minimizing occlusions. Additionally, ActiveVLA applies a 3D zoom-in to improve resolution in key areas. Together, these steps enable finer-grained active perception for precise manipulation. Extensive experiments demonstrate that ActiveVLA achieves precise 3D manipulation and outperforms state-of-the-art baselines on three simulation benchmarks. Moreover, ActiveVLA transfers seamlessly to real-world scenarios, enabling robots to learn high-precision tasks in complex environments.

Figures (8)

• Figure 1: Comparison between previous VLA methods and ActiveVLA. Traditional VLA systems often fail in tasks like “bring the apples on the table” because their fixed cameras miss critical details or become occluded. In contrast, ActiveVLA leverages 3D scene understanding to freely place virtual cameras and synthesize optimal viewpoints, enabling robots to adjust their view for clearer, more informative observations and thus achieve more reliable manipulation performance even under occlusion.
• Figure 2: The pipeline of ActiveVLA.ActiveVLA is a 3D vision-language-action framework that adopts a two-stage, coarse-to-fine strategy. In the coarse stage, three orthographic projections of the 3D scene and a language instruction are processed by the PaliGemma backbone to generate 2D heatmaps, which are then back-projected to locate the most relevant 3D region. In the fine stage, an active perception module selects new views and performs a 3D zoom-in on this region. The refined PaliGemma then predicts heatmaps for key end-effector positions, while an action decoder outputs the final 3D action.
• Figure 3: Qualitative results of fine-grained manipulation tasks. Left of the dotted line (coarse stage): (a) project 3D modalities onto orthographic images, then (b) predict heatmaps to mark critical regions. Right of the dotted line (fine stage): using these regions, perform (c) active view selection and (d) active 3D zoom-in for fine-grained manipulation in complex scenes.
• Figure 4: Visualization of ActiveVLA in complex manipulation tasks. It actively perceives and precisely completes tasks despite severe occlusions and complex spatial structures.
• Figure 5: Success rates of ActiveVLA under different hyperparameters: (a) Number of selected views; (b) Active 3D zoom-in factor. Experiments are evaluated on the RLBench benchmark.