Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spatial RoboGrasp: Generalized Robotic Grasping Control Policy

Yiqi Huang, Travis Davies, Jiahuan Yan, Jiankai Sun, Xiang Chen, Luhui Hu

TL;DR

This work tackles the robustness and generalization gaps in robotic grasping by integrating a spatially grounded perception stack with a diffusion-based policy. Spatial RoboGrasp combines AugFusion domain-randomized RGB augmentations, monocular depth estimation, and 6-DoF grasp prompts to produce a rich observation embedding that conditions a stochastic, contact-aware controller. Ablation and task-based evaluations across PickBig, PickCup, and PickGoods demonstrate substantial improvements in task and grasp success under environmental variation, with depth, augmentation, and grasp prompts each contributing uniquely to performance. The approach promises scalable, real-world deployment by avoiding heavy 3D sensing while delivering reliable, goal-directed manipulation in unstructured settings.

Abstract

Achieving generalizable and precise robotic manipulation across diverse environments remains a critical challenge, largely due to limitations in spatial perception. While prior imitation-learning approaches have made progress, their reliance on raw RGB inputs and handcrafted features often leads to overfitting and poor 3D reasoning under varied lighting, occlusion, and object conditions. In this paper, we propose a unified framework that couples robust multimodal perception with reliable grasp prediction. Our architecture fuses domain-randomized augmentation, monocular depth estimation, and a depth-aware 6-DoF Grasp Prompt into a single spatial representation for downstream action planning. Conditioned on this encoding and a high-level task prompt, our diffusion-based policy yields precise action sequences, achieving up to 40% improvement in grasp success and 45% higher task success rates under environmental variation. These results demonstrate that spatially grounded perception, paired with diffusion-based imitation learning, offers a scalable and robust solution for general-purpose robotic grasping.

Spatial RoboGrasp: Generalized Robotic Grasping Control Policy

TL;DR

This work tackles the robustness and generalization gaps in robotic grasping by integrating a spatially grounded perception stack with a diffusion-based policy. Spatial RoboGrasp combines AugFusion domain-randomized RGB augmentations, monocular depth estimation, and 6-DoF grasp prompts to produce a rich observation embedding that conditions a stochastic, contact-aware controller. Ablation and task-based evaluations across PickBig, PickCup, and PickGoods demonstrate substantial improvements in task and grasp success under environmental variation, with depth, augmentation, and grasp prompts each contributing uniquely to performance. The approach promises scalable, real-world deployment by avoiding heavy 3D sensing while delivering reliable, goal-directed manipulation in unstructured settings.

Abstract

Achieving generalizable and precise robotic manipulation across diverse environments remains a critical challenge, largely due to limitations in spatial perception. While prior imitation-learning approaches have made progress, their reliance on raw RGB inputs and handcrafted features often leads to overfitting and poor 3D reasoning under varied lighting, occlusion, and object conditions. In this paper, we propose a unified framework that couples robust multimodal perception with reliable grasp prediction. Our architecture fuses domain-randomized augmentation, monocular depth estimation, and a depth-aware 6-DoF Grasp Prompt into a single spatial representation for downstream action planning. Conditioned on this encoding and a high-level task prompt, our diffusion-based policy yields precise action sequences, achieving up to 40% improvement in grasp success and 45% higher task success rates under environmental variation. These results demonstrate that spatially grounded perception, paired with diffusion-based imitation learning, offers a scalable and robust solution for general-purpose robotic grasping.

Paper Structure

This paper contains 20 sections, 5 equations, 7 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology
  4. AugFusion
  5. Monocular Depth Estimation Module
  6. Grasp Prediction Module
  7. Observation Encoder
  8. Robotic Action Head
  9. Experiments and Evaluation
  10. Task Description
  11. Data Processing
  12. Evaluation Metrics
  13. Computation Resources
  14. Results and Discussion
  15. Lighting Robustness Across Exposure Conditions
  16. ...and 5 more sections

Figures (7)

  • Figure 1: An overview Spatial RoboGrasp architecture, demonstrating the integration of image augmentations, depth estimation and a grasping prompt prediction modules. These observation conditions and robot state data to enhance generalizability and precision of grasping manipulation.
  • Figure 2: Grasp prompt visualization in the PickBig task. (a) Predicted oriented 2D grasp box from the RGB image. (b) Corresponding 6-DoF grasp prompt derived using camera intrinsics and a rotation matrix.
  • Figure 3: Placement generalization setup for the PickBig task. (a) and (b) illustrate two of the eight object configurations. The goal is to identify and grasp the larger of two similarly shaped blocks along its diameter.
  • Figure 4: Few-shot PickCup setup. (a) Green mug with 5-shot handle grasping. (b) Blue plastic cup with 10-shot diameter grasping.
  • Figure 5: Promptable PickGoods setup. (a) Grasp prompt for a chocolate bar; (b) for a biscuit. The goal is to follow prompts to pick the target item.
  • ...and 2 more figures