AnyPos: Automated Task-Agnostic Actions for Bimanual Manipulation

TL;DR

The paper addresses the data bottleneck in vision-language-action manipulation by proposing a task-agnostic action paradigm and a scalable data collection pipeline (ATARA). It couples this with AnyPos, an inverse dynamics model featuring Arm-Decoupled Estimation and a Direction-Aware Decoder, and validates actions via video-grounded priors, enabling zero-shot task generalization and improved real-world manipulation via replay and diffusion-based video deployment. Key results show strong action-prediction accuracy (57.13%), high real-world replay success (92.59%), and 30–40% gains in downstream tasks, with substantially reduced data collection costs compared to human demonstrations. This framework offers a scalable path toward general-purpose embodied agents by decoupling high-level semantics from low-level motor control and leveraging video priors for grounding.

Abstract

Vision-language-action (VLA) models have shown promise on task-conditioned control in complex settings such as bimanual manipulation. However, the heavy reliance on task-specific human demonstrations limits their generalization and incurs high data acquisition costs. In this work, we present a new notion of task-agnostic action paradigm that decouples action execution from task-specific conditioning, enhancing scalability, efficiency, and cost-effectiveness. To address the data collection challenges posed by this paradigm -- such as low coverage density, behavioral redundancy, and safety risks -- we introduce ATARA (Automated Task-Agnostic Random Actions), a scalable self-supervised framework that accelerates collection by over $ 30\times $ compared to human teleoperation. To further enable effective learning from task-agnostic data, which often suffers from distribution mismatch and irrelevant trajectories, we propose AnyPos, an inverse dynamics model equipped with Arm-Decoupled Estimation and a Direction-Aware Decoder (DAD). We additionally integrate a video-conditioned action validation module to verify the feasibility of learned policies across diverse manipulation tasks. Extensive experiments show that the AnyPos-ATARA pipeline yields a 51% improvement in test accuracy and achieves 30-40% higher success rates in downstream tasks such as lifting, pick-and-place, and clicking, using replay-based video validation. Project Page: https://embodiedfoundation.github.io/vidar_anypos

Figures (12)

• Figure 1: Overview of AnyPos. Our efficient auto-collected task-agnostic action collection method combines AnyPos training, achieving state-of-the-art accuracy and generalizability of image-to-action regression to unseen tasks.
• Figure 2: AnyPos illustration. We obtain a task-agnostic training dataset covering the entire cubic workspace of dual robotic arms using ATARA. Input to AnyPos: An image containing the robotic arms. Output of AnyPos: The action/joint position values inferred from the image.
• Figure 3: The schematic of the dual-arm setup. The red box is added manually, not model input. The bottom-left/right subfigures display left/right grippers. The top subfigure depicts the 2 lightweight 6-DOF robotic arms, each comprising 2 base joints, 1 elbow joint, and 3 high-precision wrist joints.
• Figure 4: The results of AnyPos with video replay to accomplish various manipulation tasks.
• Figure 5: The results of AnyPos collaborating with video generation models to accomplish various manipulation tasks.