Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TWISTED-RL: Hierarchical Skilled Agents for Knot-Tying without Human Demonstrations

Guy Freund, Tom Jurgenson, Matan Sudry, Erez Karpas

TL;DR

Results demonstrate that TWISTED-RL manages to solve previously unattainable knots of higher complexity, including commonly used knots such as the Figure-8 and the Overhand, and establishes TWISTED-RL as the new state-of-the-art in robotic knot-tying without human demonstrations.

Abstract

Robotic knot-tying represents a fundamental challenge in robotics due to the complex interactions between deformable objects and strict topological constraints. We present TWISTED-RL, a framework that improves upon the previous state-of-the-art in demonstration-free knot-tying (TWISTED), which smartly decomposed a single knot-tying problem into manageable subproblems, each addressed by a specialized agent. Our approach replaces TWISTED's single-step inverse model that was learned via supervised learning with a multi-step Reinforcement Learning policy conditioned on abstract topological actions rather than goal states. This change allows more delicate topological state transitions while avoiding costly and ineffective data collection protocols, thus enabling better generalization across diverse knot configurations. Experimental results demonstrate that TWISTED-RL manages to solve previously unattainable knots of higher complexity, including commonly used knots such as the Figure-8 and the Overhand. Furthermore, the increase in success rates and drop in planning time establishes TWISTED-RL as the new state-of-the-art in robotic knot-tying without human demonstrations.

TWISTED-RL: Hierarchical Skilled Agents for Knot-Tying without Human Demonstrations

TL;DR

Results demonstrate that TWISTED-RL manages to solve previously unattainable knots of higher complexity, including commonly used knots such as the Figure-8 and the Overhand, and establishes TWISTED-RL as the new state-of-the-art in robotic knot-tying without human demonstrations.

Abstract

Robotic knot-tying represents a fundamental challenge in robotics due to the complex interactions between deformable objects and strict topological constraints. We present TWISTED-RL, a framework that improves upon the previous state-of-the-art in demonstration-free knot-tying (TWISTED), which smartly decomposed a single knot-tying problem into manageable subproblems, each addressed by a specialized agent. Our approach replaces TWISTED's single-step inverse model that was learned via supervised learning with a multi-step Reinforcement Learning policy conditioned on abstract topological actions rather than goal states. This change allows more delicate topological state transitions while avoiding costly and ineffective data collection protocols, thus enabling better generalization across diverse knot configurations. Experimental results demonstrate that TWISTED-RL manages to solve previously unattainable knots of higher complexity, including commonly used knots such as the Figure-8 and the Overhand. Furthermore, the increase in success rates and drop in planning time establishes TWISTED-RL as the new state-of-the-art in robotic knot-tying without human demonstrations.
Paper Structure (30 sections, 3 figures, 1 table)

This paper contains 30 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. TWISTED
  4. Reinforcement Learning (RL)
  5. Problem Formulation
  6. Method
  7. From Monolithic Policies to Specialized Agents
  8. Learning Policies Dependent on High-Level Action Skills
  9. Multi-Step Episodes
  10. Goal and Initial Low-Level State Selection
  11. Experiments
  12. Experimental Setup
  13. Results
  14. Related Works
  15. Limitations
  16. ...and 15 more sections

Figures (3)

  • Figure 1: The bottom row shows the Figure-8 knot (left) and the Overhand knot (right) performed by TWISTED-RL. The top row shows the complexity of the rope measured in crossing number, and the transitions are labeled with the corresponding Reidemeister moves.
  • Figure 2: Illustration of TWISTED-RL System Architecture. Top row: the unmodified TWISTED components showing the reachable configurations and high-level planning. Bottom row: TWISTED-RL low-level execution with three episodes showing progression between crossing numbers. First episode: $0\rightarrow1$ with R1 action, second episode: $1\rightarrow2$ with Cross action, third episode: $2\rightarrow3$ with R1 action, each using its specialized agent. Red arrows indicate MuJoCo simulation steps.
  • Figure 3: Left: Anytime performance analysis of TWISTED and TWISTED-RL variants on the 3-Easy test set. Right: Anytime performance analysis of TWISTED-RL-C across all test sets. The X-axis is the runtime in seconds, and the Y-axis is the cumulative success rate (%).