Planar Robot Casting with Real2Sim2Real Self-Supervised Learning

TL;DR

This work tackles Planar Robot Casting (PRC), where a single dynamic wrist motion moves a cable's free end toward a target beyond the robot's reach. It introduces Real2Sim2Real (R2S2R), a self-supervised pipeline that collects physical trajectories, tunes simulators to real data via Differential Evolution, generates vast simulated data, and trains policies from a weighted mix of real and simulated samples. The approach yields PRC policies with median endpoint errors between 8% and 14% of cable length, outperforming baselines and single-source data policies across three cables and multiple simulators, highlighting the practicality of simulator-tuned data efficiency for deformable-object control. The work demonstrates that modest physical data, when combined with tuned physics simulators, can substantially close the Sim2Real gap for dynamic cable manipulation and offers pathways to extend to 3D, other deformables, and additional manipulation tasks.

Abstract

This paper introduces the task of {\em Planar Robot Casting (PRC)}: where one planar motion of a robot arm holding one end of a cable causes the other end to slide across the plane toward a desired target. PRC allows the cable to reach points beyond the robot workspace and has applications for cable management in homes, warehouses, and factories. To efficiently learn a PRC policy for a given cable, we propose Real2Sim2Real, a self-supervised framework that automatically collects physical trajectory examples to tune parameters of a dynamics simulator using Differential Evolution, generates many simulated examples, and then learns a policy using a weighted combination of simulated and physical data. We evaluate Real2Sim2Real with three simulators, Isaac Gym-segmented, Isaac Gym-hybrid, and PyBullet, two function approximators, Gaussian Processes and Neural Networks (NNs), and three cables with differing stiffness, torsion, and friction. Results with 240 physical trials suggest that the PRC policies can attain median error distance (as % of cable length) ranging from 8% to 14%, outperforming baselines and policies trained on only real or only simulated examples. Code, data, and videos are available at https://tinyurl.com/robotcast.

Figures (6)

• Figure 1: In Planar Robot Casting (PRC), a single dynamic planar motion of a robot wrist holding one end of a cable causes the other end of the cable to slide across the plane and stop near a desired target point, which may lie outside the robot workspace. (A) shows a side view of a UR5 robot with cable and planar workspace, (B) illustrates test performance on Cable 1 for 5 trials (dots) on 16 targets (stars). The gold inner sector represents the robot workspace, while the grey outer sector represents the reachable workspace of the cable. (C,D) each show five superimposed overhead views of the robot and cable with associated target points after PRC actions with the learned policy, in (C) an example with low error and in (D) an example with high error in endpoint position.
• Figure 2: The Real2Sim2Real pipeline for PRC. We collect a physical dataset $\mathcal{D}_\textrm{phys}$ (A) in a self-supervised manner. We subsample $\mathcal{D}_\textrm{phys}$ to generate $\mathcal{D}_\textrm{tune}$, and use it to tune simulation parameters so that its trajectories match real trajectories using Differential Evolution (B), then use the tuned simulator to generate a large dataset $\mathcal{D}_\textrm{sim}$ (C). We use a weighted combination of $\mathcal{D}_\textrm{sim}$ and $\mathcal{D}_\textrm{phys}$ to train the policy (D) and evaluate the policy in real (E).
• Figure 3: Example of a spline trajectory traced by the end effector of the UR5 for $r_0=0.6$. The reset procedure brings the end effector to $(r_0, 0)$ and the motion is smoothly interpolated along one spline to $(r_1, \theta_1)$, then along a second spline to $(r_2, \theta_2)$. Along the second spline, an offset $\alpha$ is added to the wrist joint angle produced by the IK solver.
• Figure 4: Three cable simulation models for Real2Sim tuning. From top to bottom: the PyBullet model consists of capsule rigid bodies connected by 6 DOF spring constraints. The hybrid model is a soft-body rod connected to a capsule rigid body at the endpoint. The segmented model also consists of a string of capsule rigid bodies, but is connected using ball joints.
• Figure 5: Three cables used in physical experiments. Cable 1 is a thin blue paracord, Cable 2 is a nylon cable, and Cable 3 is a thick jump rope. Each endpoint has an attached mass. The respective cable lengths are 0.63m, 0.65m, and 0.65m, the respective masses are 8g, 50g, and 45g, and the respective radii are 4.5mm, 10mm, and 14mm.