Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Collapse and Collision Aware Grasping for Cluttered Shelf Picking

Abhinav Pathak, Rajkumar Muthusamy

TL;DR

This work tackles safe cluttered-shelf retrieval by integrating physics reasoning into grasp planning. It introduces a real-to-sim pipeline that reconstructs an approximate scene from a single RGB-D image and uses PyBullet to simulate collapses and collisions before execution. The authors compare a physics-based planning approach with a heuristic baseline for both single-box extraction and shelf clearance, reporting substantial improvements in efficiency and success rates, especially in unstructured stacks. The results suggest that physics-aware planning enhances reliability and throughput for warehouse manipulation, with future work aiming to broaden sensing and manipulation capabilities.

Abstract

Efficient and safe retrieval of stacked objects in warehouse environments is a significant challenge due to complex spatial dependencies and structural inter-dependencies. Traditional vision-based methods excel at object localization but often lack the physical reasoning required to predict the consequences of extraction, leading to unintended collisions and collapses. This paper proposes a collapse and collision aware grasp planner that integrates dynamic physics simulations for robotic decision-making. Using a single image and depth map, an approximate 3D representation of the scene is reconstructed in a simulation environment, enabling the robot to evaluate different retrieval strategies before execution. Two approaches 1) heuristic-based and 2) physics-based are proposed for both single-box extraction and shelf clearance tasks. Extensive real-world experiments on structured and unstructured box stacks, along with validation using datasets from existing databases, show that our physics-aware method significantly improves efficiency and success rates compared to baseline heuristics.

Collapse and Collision Aware Grasping for Cluttered Shelf Picking

TL;DR

This work tackles safe cluttered-shelf retrieval by integrating physics reasoning into grasp planning. It introduces a real-to-sim pipeline that reconstructs an approximate scene from a single RGB-D image and uses PyBullet to simulate collapses and collisions before execution. The authors compare a physics-based planning approach with a heuristic baseline for both single-box extraction and shelf clearance, reporting substantial improvements in efficiency and success rates, especially in unstructured stacks. The results suggest that physics-aware planning enhances reliability and throughput for warehouse manipulation, with future work aiming to broaden sensing and manipulation capabilities.

Abstract

Efficient and safe retrieval of stacked objects in warehouse environments is a significant challenge due to complex spatial dependencies and structural inter-dependencies. Traditional vision-based methods excel at object localization but often lack the physical reasoning required to predict the consequences of extraction, leading to unintended collisions and collapses. This paper proposes a collapse and collision aware grasp planner that integrates dynamic physics simulations for robotic decision-making. Using a single image and depth map, an approximate 3D representation of the scene is reconstructed in a simulation environment, enabling the robot to evaluate different retrieval strategies before execution. Two approaches 1) heuristic-based and 2) physics-based are proposed for both single-box extraction and shelf clearance tasks. Extensive real-world experiments on structured and unstructured box stacks, along with validation using datasets from existing databases, show that our physics-aware method significantly improves efficiency and success rates compared to baseline heuristics.

Paper Structure

This paper contains 17 sections, 1 equation, 12 figures, 3 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Real-to-Sim Pipeline
  5. Simulation Properties and Considerations
  6. Collapse Detection and Simulating Box Removal
  7. Single-Box Removal Using the Physics-Aware Approach
  8. Shelf Clearance Using the Dynamic Simulation
  9. Heuristic-Based Sequence Planning
  10. Experiments and Results
  11. Experimental Setup
  12. Considered Stacking Complexities
  13. Training Settings
  14. Results
  15. Scalability
  16. ...and 2 more sections

Figures (12)

  • Figure 1: Cluttered shelf object retrieval (a) The autonomous robot observes the stacking complexities (b) using the percieved image and its features the robot grasp planner conducts physics driven simulation to predict collapses and collison for the retrieval tasks (c) grasp sequence for desired box extraction and clearing out all boxes from shelf
  • Figure 2: Overview of the proposed grasping pipeline using the physics-aware approach for safe cardboard box extraction
  • Figure 3: Illustration showcasing how an input image is converted to a simulation scene
  • Figure 4: Collapse detection during box removal, highlighting the impact when Box B2 is removed.
  • Figure 6: Illustration showing first iteration of the shelf clearing algorithm
  • ...and 7 more figures