CADGrasp: Learning Contact and Collision Aware General Dexterous Grasping in Cluttered Scenes

TL;DR

CADGrasp, a two-stage algorithm for general dexterous grasping using single-view point cloud inputs, is proposed, demonstrating its capability to mitigate collisions while maintaining a high grasp success rate across diverse objects and complex scenes.

Abstract

Dexterous grasping in cluttered environments presents substantial challenges due to the high degrees of freedom of dexterous hands, occlusion, and potential collisions arising from diverse object geometries and complex layouts. To address these challenges, we propose CADGrasp, a two-stage algorithm for general dexterous grasping using single-view point cloud inputs. In the first stage, we predict sparse IBS, a scene-decoupled, contact- and collision-aware representation, as the optimization target. Sparse IBS compactly encodes the geometric and contact relationships between the dexterous hand and the scene, enabling stable and collision-free dexterous grasp pose optimization. To enhance the prediction of this high-dimensional representation, we introduce an occupancy-diffusion model with voxel-level conditional guidance and force closure score filtering. In the second stage, we develop several energy functions and ranking strategies for optimization based on sparse IBS to generate high-quality dexterous grasp poses. Extensive experiments in both simulated and real-world settings validate the effectiveness of our approach, demonstrating its capability to mitigate collisions while maintaining a high grasp success rate across diverse objects and complex scenes.

Figures (9)

• Figure 1: We propose CADGrasp, which learns a contact- and collision-aware intermediate representation as a constraint, and further obtains the dexterous grasp pose with an optimization method to achieve single-view dexterous hand grasping in cluttered scenes.
• Figure 2: Overview of CADGrasp, a two-stage framework for dexterous grasping in cluttered scenes. (I) Conditional IBS Generation: A diffusion model is trained to model the conditional probability distribution $p(\mathcal{I} | \mathcal{P}, \mathbf{T})$. (II) Grasp Pose Optimization: We optimize the grasp poses $\mathcal{G}$ with predicted sparse IBS $\hat{\mathcal{I}}$ as constraints.
• Figure 3: Creation of the sparse IBS for dexterous grasping. Given a cluttered scene, we first generate the grasp pose $\mathbf{g}$ using an optimization algorithm. Then, we canonicalize and crop the scene point cloud $\mathcal{P}$ to obtain the canonicalized point cloud $\mathcal{P}^{*}$. Finally, we compute the sparse IBS $\mathcal{I}$ based on the canonicalized point cloud.
• Figure 4: IBS generation. We train a conditional occupancy-diffusion model to model the conditional probability distribution $p(\mathcal{I}|\mathcal{P}^{*})$, where $\mathcal{P}^{*}$ is the canonicalized and voxelized point cloud. The voxel-level alignment provides hierarchical conditions during the denoising process.
• Figure 5: Real-world experiment setup.