Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cooperative Grasping and Transportation using Multi-agent Reinforcement Learning with Ternary Force Representation

Ing-Sheng Bernard-Tiong, Yoshihisa Tsurumine, Ryosuke Sota, Kazuki Shibata, Takamitsu Matsubara

TL;DR

This study proposes multi-agent reinforcement learning (MARL) with ternary force representation, a force representation that maintains consistent representation against variations in grasping environment, and demonstrates the robustness of the proposed method to changes in grasping force, object size and geometry as well as inherent sim2real gap.

Abstract

Cooperative grasping and transportation require effective coordination to complete the task. This study focuses on the approach leveraging force-sensing feedback, where robots use sensors to detect forces applied by others on an object to achieve coordination. Unlike explicit communication, it avoids delays and interruptions; however, force-sensing is highly sensitive and prone to interference from variations in grasping environment, such as changes in grasping force, grasping pose, object size and geometry, which can interfere with force signals, subsequently undermining coordination. We propose multi-agent reinforcement learning (MARL) with ternary force representation, a force representation that maintains consistent representation against variations in grasping environment. The simulation and real-world experiments demonstrate the robustness of the proposed method to changes in grasping force, object size and geometry as well as inherent sim2real gap.

Cooperative Grasping and Transportation using Multi-agent Reinforcement Learning with Ternary Force Representation

TL;DR

This study proposes multi-agent reinforcement learning (MARL) with ternary force representation, a force representation that maintains consistent representation against variations in grasping environment, and demonstrates the robustness of the proposed method to changes in grasping force, object size and geometry as well as inherent sim2real gap.

Abstract

Cooperative grasping and transportation require effective coordination to complete the task. This study focuses on the approach leveraging force-sensing feedback, where robots use sensors to detect forces applied by others on an object to achieve coordination. Unlike explicit communication, it avoids delays and interruptions; however, force-sensing is highly sensitive and prone to interference from variations in grasping environment, such as changes in grasping force, grasping pose, object size and geometry, which can interfere with force signals, subsequently undermining coordination. We propose multi-agent reinforcement learning (MARL) with ternary force representation, a force representation that maintains consistent representation against variations in grasping environment. The simulation and real-world experiments demonstrate the robustness of the proposed method to changes in grasping force, object size and geometry as well as inherent sim2real gap.

Paper Structure

This paper contains 27 sections, 1 equation, 5 figures, 5 tables.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. Cooperative Object Transportation
  4. MARL for Cooperative Transportation
  5. Method
  6. Decentralized Partially Observable Markov Decision Processes (Dec-POMDP)
  7. Proposed Framework
  8. Ternary Force Representation
  9. Asymmetric Actor-critic with Ternary Force Representation
  10. Reward Design
  11. Policy Optimization
  12. Simulation Experiment
  13. Experiment Overview
  14. Experiment setup
  15. Baseline
  16. ...and 12 more sections

Figures (5)

  • Figure 2: Cooperative grasping and transportation by two HSRs through implicit communication via force-sensing feedback.
  • Figure 3: Asysmetric actor-critic-based MARL framework with ternary force representation. Even though the raw force signals differ between training and execution, converting raw force signals to the representation with consistent values of -1, 0, or 1 ensures consistency and retains informative values despite variations in the grasping environment.
  • Figure 4: Simulation environment for cooperative grasping and transportation.
  • Figure 5: Cumulative rewards of evaluated methods.
  • Figure 6: Successive frames of two HSRs trained by our method successfully grasp and transport an object to the target position. (1) Initially, the two robots are placed randomly around the object. (2) The robots approach their respective grasping points. (3) Robot 2 successfully grasps the object first and waits for Robot 1 to complete its grasp. (4) Robot 1 then grasps the object. The robot that first grasps the object detects its partner’s grasp by observing changes in the force signals and the orientation of the object. (5) After both robots have securely grasped the object, they lift it together. (6-7) They transport the object toward the target, coordinating through ternary force representation. (8) Finally, the object reaches the target position.