ShakingBot: Dynamic Manipulation for Bagging

TL;DR

ShakingBot addresses the challenging problem of robotic bagging with deformable plastic bags by applying dynamic manipulation guided by a perception-driven policy. It introduces three novel action primitives—Bag Adjustment, Dual-arm Shaking, and One-arm Holding—together with a region perception module based on semantic segmentation to locate bag handles and the opening rim. The approach achieves higher efficiency and opening success than quasi-static baselines, demonstrated on a dual-UR5 setup, with generalization across bag sizes, colors, and patterns and a reported 21/33 success for inserting at least one item. This work highlights the practical value of dynamic manipulation for deformable containers and provides a framework that can extend to other bag-like or deformable objects in real-world applications.

Abstract

Bag manipulation through robots is complex and challenging due to the deformability of the bag. Based on dynamic manipulation strategy, we propose a new framework, ShakingBot, for the bagging tasks. ShakingBot utilizes a perception module to identify the key region of the plastic bag from arbitrary initial configurations. According to the segmentation, ShakingBot iteratively executes a novel set of actions, including Bag Adjustment, Dual-arm Shaking, and One-arm Holding, to open the bag. The dynamic action, Dual-arm Shaking, can effectively open the bag without the need to account for the crumpled configuration.Then, we insert the items and lift the bag for transport. We perform our method on a dual-arm robot and achieve a success rate of 21/33 for inserting at least one item across various initial bag configurations. In this work, we demonstrate the performance of dynamic shaking actions compared to the quasi-static manipulation in the bagging task. We also show that our method generalizes to variations despite the bag's size, pattern, and color.

Figures (13)

• Figure 1: Opening the bag with dynamic actions. (1) Initial highly unstructured bag and two solid objects. (2) Through region perception, the robot grasps the two handles and adjusts the distance between the two arms. (3) The arms shake the bag at high speed according to the pre-defined trajectory, which makes the air into the bag. (4) One arm holds the opened bag on the workspace. (5) The two arms lift the bag filled with the inserted items
• Figure 2: Overview of ShakingBot. See the left of the figure. The robot starts with an unstructured bag and two items. As shown in the flow, ShakingBot opens the bag according to the steps shown (see Section \ref{['shakingbot_bagging']} for details). When the bag opening metric exceeds a certain threshold, ShakingBot proceeds to the item insertion stage. If the robot lifts the bag with all the items inside, the trial is a complete success
• Figure 3: Left: Various plastic bags adopted to train and test the region perception module. The bags include different sizes, patterns, and different colors. Right: A bag with red paint on its handles and green paint around its rim. The paint color can be changed into others according to the pattern color of the bag
• Figure 4: Pipeline for our method: The perception module takes depth images and outputs segmentation masks for the bag handles and rim. The robot grasps the key points and executes dynamic actions. Last, the robot inserts the items and lifts the bag
• Figure 5: The architecture of Deeplabv3+