ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning

TL;DR

ManipTrans is introduced, a novel two-stage method for efficiently transferring human bimanual skills to dexterous robotic hands in simulation, which surpasses state-of-the-art methods in success rate, fidelity, and efficiency.

Abstract

Human hands play a central role in interacting, motivating increasing research in dexterous robotic manipulation. Data-driven embodied AI algorithms demand precise, large-scale, human-like manipulation sequences, which are challenging to obtain with conventional reinforcement learning or real-world teleoperation. To address this, we introduce ManipTrans, a novel two-stage method for efficiently transferring human bimanual skills to dexterous robotic hands in simulation. ManipTrans first pre-trains a generalist trajectory imitator to mimic hand motion, then fine-tunes a specific residual module under interaction constraints, enabling efficient learning and accurate execution of complex bimanual tasks. Experiments show that ManipTrans surpasses state-of-the-art methods in success rate, fidelity, and efficiency. Leveraging ManipTrans, we transfer multiple hand-object datasets to robotic hands, creating DexManipNet, a large-scale dataset featuring previously unexplored tasks like pen capping and bottle unscrewing. DexManipNet comprises 3.3K episodes of robotic manipulation and is easily extensible, facilitating further policy training for dexterous hands and enabling real-world deployments.

Figures (11)

• Figure 1: ManipTrans for Bimanual Dexterous Manipulations. Retargeting methods often struggle with transferring MoCap data to physically plausible motions, while our ManipTrans efficiently produces task-compliant, physically accurate motions. It also generalizes across embodiments like Inspire hands inspirehandurl, Shadow hands shadowhandurl, and articulated MANO hands MANO:SIGGRAPHASIA:2017christen2022d.
• Figure 2: Our ManipTrans Pipeline. We first pre-train a hand motion imitation model with large-scale human demonstrations, then fine-tune a residual policy to adapt to task-specific physical constraints.
• Figure 3: Qualitative Results of ManipTrans. We showcase the transfer results using the Inspire left and right hands on both single-hand tasks (top two rows) and bimanual tasks (bottom row) from the OakInk-V2 zhan2024oakink2 dataset. Notably, the dexterous hands successfully manipulate delicate and slim objects, such as a pen and a flower stem.
• Figure 4: Qualitative Comparison with QuasiSim liu2025parameterized.ManipTrans produces more natural motion of the Shadow hand (purple region) and is applicable to other dexterous hands.
• Figure 5: Cross Embodiments Results: Putting off Alcohol lamp.