Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DeRi-IGP: Learning to Manipulate Rigid Objects Using Deformable Objects via Iterative Grasp-Pull

Zixing Wang, Ahmed H. Qureshi

TL;DR

DeRi-IGP tackles the challenge of manipulating rigid objects through deformable ropes by learning a vision-based, model-free policy that parameterizes an Iterative Grasp-Pull primitive. The framework integrates three neural modules—Grasping Point Network, Pulling Point Network, and Delta Position Network—along with two subgoal planners and a multiagent workflow to generate and evaluate action proposals, enabling robust, long-horizon manipulation. Empirical results in simulation and real-world settings show that DeRi-IGP outperforms model-based and learning-based baselines, demonstrates strong sim-to-real transfer, and supports human–robot collaboration and distant object acquisition. The work broadens the operational space and generalization capacity for heterogeneous soft-rigid manipulation, with practical implications for transport and manipulation tasks in cluttered or uncertain environments.

Abstract

Robotic manipulation of rigid objects via deformable linear objects (DLO) such as ropes is an emerging field of research with applications in various rigid object transportation tasks. A few methods that exist in this field suffer from limited robot action and operational space, poor generalization ability, and expensive model-based development. To address these challenges, we propose a universally applicable moving primitive called Iterative Grasp-Pull (IGP). We also introduce a novel vision-based neural policy that learns to parameterize the IGP primitive to manipulate DLO and transport their attached rigid objects to the desired goal locations. Additionally, our decentralized algorithm design allows collaboration among multiple agents to manipulate rigid objects using DLO. We evaluated the effectiveness of our approach in both simulated and real-world environments for a variety of soft-rigid body manipulation tasks. In the real world, we also demonstrate the effectiveness of our decentralized approach through human-robot collaborative transportation of rigid objects to given goal locations. We also showcase the large operational space of IGP primitive by solving distant object acquisition tasks. Lastly, we compared our approach with several model-based and learning-based baseline methods. The results indicate that our method surpasses other approaches by a significant margin.

DeRi-IGP: Learning to Manipulate Rigid Objects Using Deformable Objects via Iterative Grasp-Pull

TL;DR

DeRi-IGP tackles the challenge of manipulating rigid objects through deformable ropes by learning a vision-based, model-free policy that parameterizes an Iterative Grasp-Pull primitive. The framework integrates three neural modules—Grasping Point Network, Pulling Point Network, and Delta Position Network—along with two subgoal planners and a multiagent workflow to generate and evaluate action proposals, enabling robust, long-horizon manipulation. Empirical results in simulation and real-world settings show that DeRi-IGP outperforms model-based and learning-based baselines, demonstrates strong sim-to-real transfer, and supports human–robot collaboration and distant object acquisition. The work broadens the operational space and generalization capacity for heterogeneous soft-rigid manipulation, with practical implications for transport and manipulation tasks in cluttered or uncertain environments.

Abstract

Robotic manipulation of rigid objects via deformable linear objects (DLO) such as ropes is an emerging field of research with applications in various rigid object transportation tasks. A few methods that exist in this field suffer from limited robot action and operational space, poor generalization ability, and expensive model-based development. To address these challenges, we propose a universally applicable moving primitive called Iterative Grasp-Pull (IGP). We also introduce a novel vision-based neural policy that learns to parameterize the IGP primitive to manipulate DLO and transport their attached rigid objects to the desired goal locations. Additionally, our decentralized algorithm design allows collaboration among multiple agents to manipulate rigid objects using DLO. We evaluated the effectiveness of our approach in both simulated and real-world environments for a variety of soft-rigid body manipulation tasks. In the real world, we also demonstrate the effectiveness of our decentralized approach through human-robot collaborative transportation of rigid objects to given goal locations. We also showcase the large operational space of IGP primitive by solving distant object acquisition tasks. Lastly, we compared our approach with several model-based and learning-based baseline methods. The results indicate that our method surpasses other approaches by a significant margin.
Paper Structure (25 sections, 3 equations, 3 figures, 2 tables)

This paper contains 25 sections, 3 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Soft-rigid Body System Manipulation
  4. Soft Body Manipulation
  5. Methods
  6. State-space Representation
  7. Action-space Representation
  8. Action Generation Module
  9. Residual Action-Outcome Prediction Module
  10. Subgoal Planner
  11. Multiagent Workflow
  12. Neural Network Architecture
  13. Data Generation
  14. Training and Testing Details
  15. Experiments
  16. ...and 10 more sections

Figures (3)

  • Figure 1: In the DeRi-IGP process, the subgoal planner assigns local targets to agents, while the GPN and PPN predict grasping and pulling points based on observations. These predicted actions are then used by the DPN to forecast the object's future position and select the best action that minimizes the object-target distance. It's important to note that all proposed robot actions will be compared for selection. The subgoal planner's map presented in the left-most sub-figure is also used as a mask on the spatial relative position map and the output of DPN.
  • Figure 2: An example of dual-bot random-position goal-reaching task. After six IGP actions, the rigid object (brown block) is moved to the target position (pink circle). Note that the task configuration is under-actuated given the relative position between the rigid object and the target, and yet DeRi-IGP finishes the task with multi-robot cooperation
  • Figure 3: A demonstration of the long-horizon human-robot collaborative goal-reaching task. The human operator and the robot take turns manipulating the rope to move the object (brown card box) to the target position marked in blue. Note, the robot failed to pick up the rope in Action 4.