Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sim2Real2Sim: Bridging the Gap Between Simulation and Real-World in Flexible Object Manipulation

Peng Chang, Taskin Padir

TL;DR

The paper addresses the persistent gap between simulation and reality in flexible object manipulation by proposing Sim2Real2Sim, a three-stage loop that starts with a rough simulation, tests and collects real-world data, and then updates the simulation models accordingly. A novel inverse-dynamics–based identification method is developed to calibrate a Piecewise-Constant-Curvature (PCC) cable model for the DRC Plug Task, with a detailed visual servoing framework enabling autonomous execution. The cable is modeled as a 15-link PCC chain with a 10 cm plug, and the dynamics are captured by $M oldsymbol{ ddot{q}} + C oldsymbol{ qd} + G + J^T oldsymbol{f}_{ext} + K oldsymbol{q} + D oldsymbol{ qd} = oldsymbol{ au}$, from which $K$ and $D$ are estimated via $K = (oldsymbol{ au} - (M oldsymbol{ ddot{q}} + C oldsymbol{ qd} + G + J^T oldsymbol{f}_{ext} + D oldsymbol{ qd})) oldsymbol{q}^{+}$ and $D = (oldsymbol{ au} - (M oldsymbol{ ddot{q}} + C oldsymbol{ qd} + G + J^T oldsymbol{f}_{ext} + K oldsymbol{q})) oldsymbol{ qd}^{+}$. The approach is validated by automating the DRC Plug Task in both simulation and real-world settings, with results showing close agreement of cable deformation (joint angles within a few milliradians and sagging angles within a few hundredths of a radian) and successful task completion, demonstrating practical impact for robust, transferable manipulation of flexible objects.

Abstract

This paper addresses a new strategy called Simulation-to-Real-to-Simulation (Sim2Real2Sim) to bridge the gap between simulation and real-world, and automate a flexible object manipulation task. This strategy consists of three steps: (1) using the rough environment with the estimated models to develop the methods to complete the manipulation task in the simulation; (2) applying the methods from simulation to real-world and comparing their performance; (3) updating the models and methods in simulation based on the differences between the real world and the simulation. The Plug Task from the 2015 DARPA Robotics Challenge Finals is chosen to evaluate our Sim2Real2Sim strategy. A new identification approach for building the model of the linear flexible objects is derived from real-world to simulation. The automation of the DRC plug task in both simulation and real-world proves the success of the Sim2Real2Sim strategy. Numerical experiments are implemented to validate the simulated model.

Sim2Real2Sim: Bridging the Gap Between Simulation and Real-World in Flexible Object Manipulation

TL;DR

The paper addresses the persistent gap between simulation and reality in flexible object manipulation by proposing Sim2Real2Sim, a three-stage loop that starts with a rough simulation, tests and collects real-world data, and then updates the simulation models accordingly. A novel inverse-dynamics–based identification method is developed to calibrate a Piecewise-Constant-Curvature (PCC) cable model for the DRC Plug Task, with a detailed visual servoing framework enabling autonomous execution. The cable is modeled as a 15-link PCC chain with a 10 cm plug, and the dynamics are captured by , from which and are estimated via and . The approach is validated by automating the DRC Plug Task in both simulation and real-world settings, with results showing close agreement of cable deformation (joint angles within a few milliradians and sagging angles within a few hundredths of a radian) and successful task completion, demonstrating practical impact for robust, transferable manipulation of flexible objects.

Abstract

This paper addresses a new strategy called Simulation-to-Real-to-Simulation (Sim2Real2Sim) to bridge the gap between simulation and real-world, and automate a flexible object manipulation task. This strategy consists of three steps: (1) using the rough environment with the estimated models to develop the methods to complete the manipulation task in the simulation; (2) applying the methods from simulation to real-world and comparing their performance; (3) updating the models and methods in simulation based on the differences between the real world and the simulation. The Plug Task from the 2015 DARPA Robotics Challenge Finals is chosen to evaluate our Sim2Real2Sim strategy. A new identification approach for building the model of the linear flexible objects is derived from real-world to simulation. The automation of the DRC plug task in both simulation and real-world proves the success of the Sim2Real2Sim strategy. Numerical experiments are implemented to validate the simulated model.

Paper Structure

This paper contains 8 sections, 5 equations, 13 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. PROBLEM STATEMENT
  3. METHODOLOGY
  4. Simulation to real-world
  5. Real-world to simulation
  6. Visual servoing in simulation and real-world
  7. EXPERIMENTAL VALIDATION
  8. CONCLUSIONS

Figures (13)

  • Figure 1: The DRC Plug Task setup in real-world.
  • Figure 2: Sim2Real2Sim flowchart representation.
  • Figure 3: Simulation environment.
  • Figure 4: Cable model design.
  • Figure 5: Geometric model of the cable. Green squares represent the sampling points, adjacent points are connected by blue lines, and vertical red lines are marked to illustrate the locations of the sampling points.
  • ...and 8 more figures