Cross-embodied Co-design for Dexterous Hands

TL;DR

The paper presents a cross-embodiment co-design framework that jointly optimizes robot hand morphology and morphology-conditioned control policies to tackle dexterous manipulation. By combining grammar-based hand generation, graph-encoded design representations, and a cross-embodiment policy trained across design families, the approach enables rapid sim-to-real design evaluation and fabrication of functional hands within 24 hours. Key findings show morphology dominates manipulation performance, with zero-shot real-world rotation demonstrated on unseen objects and multiple hands, validating the practicality of the end-to-end pipeline. The work advances dexterous robotics by enabling scalable, task-driven hand design that bridges simulation and real hardware through realistic grammars, GNN-based design embeddings, and efficient search strategies.

Abstract

Dexterous manipulation is limited by both control and design, without consensus as to what makes manipulators best for performing dexterous tasks. This raises a fundamental challenge: how should we design and control robot manipulators that are optimized for dexterity? We present a co-design framework that learns task-specific hand morphology and complementary dexterous control policies. The framework supports 1) an expansive morphology search space including joint, finger, and palm generation, 2) scalable evaluation across the wide design space via morphology-conditioned cross-embodied control, and 3) real-world fabrication with accessible components. We evaluate the approach across multiple dexterous tasks, including in-hand rotation with simulation and real deployment. Our framework enables an end-to-end pipeline that can design, train, fabricate, and deploy a new robotic hand in under 24 hours. The full framework will be open-sourced and available on our website.

Figures (16)

• Figure 1: We present our cross-embodied co-design framework for dexterous hands that jointly optimize control policies and hardware design. Four co-design hands are deployed sim-to-real.
• Figure 2: Overview of our method. Stage 1: We first randomly sample embodiments, including morphology and degrees of freedom. We then pre-train a morphology-conditioned policy across embodiments. Stage 2: The design is generated using modular grammars, with assembly rules selected based on prior performance. The cross-embodied policy is then deployed to evaluate different designs in simulation. Stage 3: Best designs and fine tuned control are then manufactured and deployed on real hardware across unseen objects.
• Figure 3: Left to Right. 1) Variable finger length of 1-10 using flat modular stacks. 2) Example embodiment with varying degrees of freedom with fingers of 3 or 4 servo motors. 3) Fingertip Variations 4) Finger number and example embodiment variations.
• Figure 4: Top Left: Three finger co-design hand, best design found by algorithm. Top Right: Five finger symmetric co-design hand. Bottom Left: Four finger co-design hand with thin fingertips. Bottom Right: Five finger anthropomorphic baseline.
• Figure 5: Top Left: Grasping task with hold time over a minute and quick grasps ($<$ 0.1 sec).Bottom: Best found design of flipping task where an object rotates along the z-axis using a fixed wrist. Bottom Right: Normalized reward of best design for Flipping, Grasping, and In-Hand Rotation.