Corner-Grasp: Multi-Action Grasp Detection and Active Gripper Adaptation for Grasping in Cluttered Environments

TL;DR

This work tackles robust grasping in cluttered bin-picking by integrating a multi-functional ReC-Gripper with active collision-avoidance and a multi-action grasp detector that jointly predicts suction and finger grasp points from a single RGB-D input. The detector is trained on a large synthetic corpus (100{,}000 images, ~2.8×10^9 grasp candidates) and a real-world set (3{,}000 images), augmented by an auxiliary electromagnetic grasping strategy to handle metallic objects. A Surface Material Detection Network (SMD-Net) guides modality selection, while a collision-avoidance planner and a grasp-fusion strategy bolster stability during movement. The approach achieves strong real-world performance, placing second in RGMC 2024 and demonstrating high success and clearance rates in table-top and bin-picking tasks, with practical impact for cluttered-environment manipulation.

Abstract

Robotic grasping is an essential capability, playing a critical role in enabling robots to physically interact with their surroundings. Despite extensive research, challenges remain due to the diverse shapes and properties of target objects, inaccuracies in sensing, and potential collisions with the environment. In this work, we propose a method for effectively grasping in cluttered bin-picking environments where these challenges intersect. We utilize a multi-functional gripper that combines both suction and finger grasping to handle a wide range of objects. We also present an active gripper adaptation strategy to minimize collisions between the gripper hardware and the surrounding environment by actively leveraging the reciprocating suction cup and reconfigurable finger motion. To fully utilize the gripper's capabilities, we built a neural network that detects suction and finger grasp points from a single input RGB-D image. This network is trained using a larger-scale synthetic dataset generated from simulation. In addition to this, we propose an efficient approach to constructing a real-world dataset that facilitates grasp point detection on various objects with diverse characteristics. Experiment results show that the proposed method can grasp objects in cluttered bin-picking scenarios and prevent collisions with environmental constraints such as a corner of the bin. Our proposed method demonstrated its effectiveness in the 9th Robotic Grasping and Manipulation Competition (RGMC) held at ICRA 2024.

Figures (14)

• Figure 1: Functionalities of ReC-Gripper: (a) Finger grasping motion; (b) Reciprocating suction motion; (c) Grasping an object using finger grasp mode; (d) Grasping an object using suction grasp mode.
• Figure 2: Adaptations of ReC-Gripper for grasping in cluttered environments: (a) Reciprocating suction cup motion to avoid potential collision sources, such as bin walls or objects near the grasping point; (b) Reconfigurable finger motion to prevent collisions when grasping in environmental constraints such as corner spaces.
• Figure 3: The gripper modification: (a) Previous gripper design with protrusions that can cause collisions; (b) Modified gripper design with a flat side to minimize potential collisions.
• Figure 4: The electromagnetic grasping strategy of our system. When the system detects a metallic object, the gripper picks up the electromagnetic holder placed near the base of the robot arm using the suction cup and moves it to the target grasping point. The electromagnetic holder then attracts the metallic object, allowing the gripper to pick it up.
• Figure 5: The proposed multi-action grasp detection network architecture. The network consists of RGB and depth trunks for shared feature extraction. Task-specific output heads predict the suction affordance map, finger affordance map, and grasp angle affordance map.