Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning to Plan & Schedule with Reinforcement-Learned Bimanual Robot Skills

Weikang Wan, Fabio Ramos, Xuning Yang, Caelan Garrett

TL;DR

The paper tackles long-horizon, contact-rich bimanual manipulation by reframing control as integrated planning and scheduling. It learns a library of RL-trained primitive skills for each arm and trains a Transformer-based high-level scheduler to output both discrete skills and continuous parameters, enabling parallel and sequential execution as needed. Across simulation experiments, the approach achieves higher success rates and more efficient coordination than end-to-end RL or sequential planners, demonstrating the value of explicit skill planning and scheduling in multi-arm manipulation. This hierarchical, planner-forward method holds promise for scalable, coordinated bimanual tasks in robotics, potentially extending to more arms and complex interactions.

Abstract

Long-horizon contact-rich bimanual manipulation presents a significant challenge, requiring complex coordination involving a mixture of parallel execution and sequential collaboration between arms. In this paper, we introduce a hierarchical framework that frames this challenge as an integrated skill planning & scheduling problem, going beyond purely sequential decision-making to support simultaneous skill invocation. Our approach is built upon a library of single-arm and bimanual primitive skills, each trained using Reinforcement Learning (RL) in GPU-accelerated simulation. We then train a Transformer-based planner on a dataset of skill compositions to act as a high-level scheduler, simultaneously predicting the discrete schedule of skills as well as their continuous parameters. We demonstrate that our method achieves higher success rates on complex, contact-rich tasks than end-to-end RL approaches and produces more efficient, coordinated behaviors than traditional sequential-only planners.

Learning to Plan & Schedule with Reinforcement-Learned Bimanual Robot Skills

TL;DR

The paper tackles long-horizon, contact-rich bimanual manipulation by reframing control as integrated planning and scheduling. It learns a library of RL-trained primitive skills for each arm and trains a Transformer-based high-level scheduler to output both discrete skills and continuous parameters, enabling parallel and sequential execution as needed. Across simulation experiments, the approach achieves higher success rates and more efficient coordination than end-to-end RL or sequential planners, demonstrating the value of explicit skill planning and scheduling in multi-arm manipulation. This hierarchical, planner-forward method holds promise for scalable, coordinated bimanual tasks in robotics, potentially extending to more arms and complex interactions.

Abstract

Long-horizon contact-rich bimanual manipulation presents a significant challenge, requiring complex coordination involving a mixture of parallel execution and sequential collaboration between arms. In this paper, we introduce a hierarchical framework that frames this challenge as an integrated skill planning & scheduling problem, going beyond purely sequential decision-making to support simultaneous skill invocation. Our approach is built upon a library of single-arm and bimanual primitive skills, each trained using Reinforcement Learning (RL) in GPU-accelerated simulation. We then train a Transformer-based planner on a dataset of skill compositions to act as a high-level scheduler, simultaneously predicting the discrete schedule of skills as well as their continuous parameters. We demonstrate that our method achieves higher success rates on complex, contact-rich tasks than end-to-end RL approaches and produces more efficient, coordinated behaviors than traditional sequential-only planners.

Paper Structure

This paper contains 12 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Problem Formulation
  5. Individual Low-Level Skills
  6. Bimanual Skill Scheduling Policy
  7. Data Generation.
  8. Policy Learning.
  9. Experiments
  10. Experiments Setup
  11. Quantitative Results
  12. Conclusion

Figures (2)

  • Figure 1: (a) Policy Overview. In our hierarchical framework, the high-level policy selects skills by predicting the skill index $k^R, k^L$ and specifying the skill parameters $\omega^R, \omega^L$ for both arms. (b) Behavior Types. Our method exhibits diverse bimanual behaviors in long-horizon tasks.
  • Figure 2: Evaluation Visualization. The long-horizon rearranging task presents diverse scenarios, such as objects being out of reach or requiring bimanual effort. Our policy handles these by planning the division of labor and collaboration between arms and correctly sequencing the necessary skills.