Learning to Design Soft Hands using Reward Models

TL;DR

Soft hands enable safe interaction but designing them to be both compliant and functional is challenging due to high-dimensional morphology–control coupling and costly evaluations. The paper presents CEM-RM, a data-driven framework that learns a design distribution over the variables $l^{fle}$, $l^{seg}$, $h_i$, $\mathbf{p}$, and $\psi$, and accelerates search by coupling Cross-Entropy Method with a neural reward model trained from teleoperation-grounded simulation rewards. Contributions include a parallel FEM-based soft-hand design space, a data-efficient optimization loop that reduces environment interactions by more than half, and hardware validation showing improved grasp success across diverse objects; ablation insights on teleoperation data and population size. Significance lies in providing a practical blueprint for data-driven co-design in soft robotics, enabling robust, rapid discovery of tendon-driven soft-hand morphologies and guiding future work on sim-to-real transfer and integrated control-design optimization.

Abstract

Soft robotic hands promise to provide compliant and safe interaction with objects and environments. However, designing soft hands to be both compliant and functional across diverse use cases remains challenging. Although co-design of hardware and control better couples morphology to behavior, the resulting search space is high-dimensional, and even simulation-based evaluation is computationally expensive. In this paper, we propose a Cross-Entropy Method with Reward Model (CEM-RM) framework that efficiently optimizes tendon-driven soft robotic hands based on teleoperation control policy, reducing design evaluations by more than half compared to pure optimization while learning a distribution of optimized hand designs from pre-collected teleoperation data. We derive a design space for a soft robotic hand composed of flexural soft fingers and implement parallelized training in simulation. The optimized hands are then 3D-printed and deployed in the real world using both teleoperation data and real-time teleoperation. Experiments in both simulation and hardware demonstrate that our optimized design significantly outperforms baseline hands in grasping success rates across a diverse set of challenging objects.

Figures (8)

• Figure 1: We present a Cross-Entropy Method (CEM) with reward model (CEM-RM) framework that optimizes block-wise, finger-wise, and tendon-routing design distributions of a soft robotic hand using pre-collected teleoperation data. Hardware experiments demonstrate that CEM-RM achieves effective design optimization with significantly fewer samples than pure optimization, enabling robust grasping of challenging objects.
• Figure 2: Soft robot hand design space. (a) 3D and side views of a finger, with the fingertip on the left and the base on the right. The tendon force $T$ is applied along the green tendon route toward the base, passing through red waypoints. Optimized parameters include segment and flexure lengths, tendon waypoint distribution, and segment thickness. (b) A three-finger soft hand (base on top, fingers pointing down), where finger orientation and mounting position are also design parameters.
• Figure 3: System overview. We first collected multiple teleoperation control datasets for each object, which are randomly sampled during optimization. The design action distribution is optimized in the CEM loop, with evaluations from both simulation and a co-trained reward model (the proportion of reward model evaluations increases smoothly during training). The action distribution ultimately converges to the optimal soft hand design.
• Figure 4: Teleoperation data collection. Human hand poses captured by the Meta Quest 3 are converted into soft-hand motion control commands for real-time teleoperation. The grasping pose, prismatic joint displacement, and tendon motion are collected and augmented in simulation.
• Figure 5: Optimized soft robot hand. We built our final design with optimized fingers, 3D-printed finger holders, an xArm mount, geared racks, and four servo motors.