Mastering Contact-rich Tasks by Combining Soft and Rigid Robotics with Imitation Learning

TL;DR

This work presents a novel hybrid robotic platform that integrates a rigid manipulator with a fully developed soft arm that is equipped with the intelligence necessary to perform flexible and generalizable tasks through imitation learning autonomously.

Abstract

Soft robots have the potential to revolutionize the use of robotic systems with their capability of establishing safe, robust, and adaptable interactions with their environment, but their precise control remains challenging. In contrast, traditional rigid robots offer high accuracy and repeatability but lack the flexibility of soft robots. We argue that combining these characteristics in a hybrid robotic platform can significantly enhance overall capabilities. This work presents a novel hybrid robotic platform that integrates a rigid manipulator with a fully developed soft arm. This system is equipped with the intelligence necessary to perform flexible and generalizable tasks through imitation learning autonomously. The physical softness and machine learning enable our platform to achieve highly generalizable skills, while the rigid components ensure precision and repeatability.

Figures (8)

• Figure 1: Overview of the proposed hybrid system combining computational and physical intelligence. Panel (A) shows a human providing a single kinesthetic demonstration by moving the rigid part of the proposed rigid-soft hybrid manipulator. The learned policy is then transported to a new task instance using keypoint-based deformation, enabling generalization without retraining. The errors introduced by the transportation process, together with possible variations in the physical surroundings (e.g., different screwdriver), are absorbed by the soft arms' physical intelligence, as described in Panel (B). The soft arm can adapt passively to interactions with objects and the ground, compensating for uncertainties and variations in the environment and in the relative positioning of the soft arm base without requiring changes to the control input. Panels (C1) and (C2) show a side-by-side comparison of demonstration and generalization for a pick-and-place task. The first column reports a summary of the two keypoints (source $\tilde{s}_i$ and target $\tilde{t}_i$) used in this task, and of the two trajectories (demonstrated in C1 and transported in C2) at the rigid robot end effector.
• Figure 2: Proposed soft–rigid hybrid platform. Panel (A) shows the integrated system, composed of a 7-DoF rigid co-bot (Franka Panda), an RGB-D camera (RealSense D405) for keypoint localization, and a cable-driven soft arm mounted at the wrist. The soft arm is actuated via two tendons by motors enclosed in the actuator case. Four buttons on the robot are used to control and log tendon references during demonstrations. Panels (B1) and (B2) illustrate the deformation capabilities of the soft arm alone when the two tendons are pulled separately (ventral bending and twisting). Panels (C1) and (C2) show the platform executing compound motions where both rigid and soft components are actuated simultaneously.
• Figure 3: Tightening a loose screw using an electric screw driver. Qualitative example of the capabilities of the proposed system. The first row shows the task execution from a perspective that allows one to appreciate the distance, while the second shows a close-up.
• Figure 4: Executions of the stacking task. Panel (A) depicts the given demonstration, and transported trajectories in different colors, where these have all been spatially aligned to the picking locations for clarity. Panel (B) shows the initial object locations and the goal stacking locations from a top-view, with the colored arrows indicating the paired locations. Panels (C$_1$-C$_3$) visualize different initial configurations and their robot executions. Note how these different configurations showcase generalization, since the task is completed in configurations that are dissimlar to that of the demonstration.
• Figure 5: Executions of the pulling task. Panel (A) shows the transported trajectories, where they have been spatially aligned by translating the picking locations to the origin for clarity. Panel (B) then shows the initial and goal locations for the object from a top-view, with the arrows indicating paired locations. Panel (C$_1$) depicts a sequence from an execution closely resembling the original demonstration, where the object is approached and dragged along a similar trajectory. Panel (C$_2$) illustrates a more demanding configuration, in which the robot must extend further to reach past the object before initiating the pull. In both instances, the manipulation is achieved without grasping, relying instead on compliant, non-prehensile contact.