T(R,O) Grasp: Efficient Graph Diffusion of Robot-Object Spatial Transformation for Cross-Embodiment Dexterous Grasping

TL;DR

Dexterous grasping across diverse hand embodiments is hampered by high-dimensional state-action spaces and representation limitations. The authors introduce the $\mathcal{T(R,O)}$ Graph, a unified, memory-efficient representation of object–robot interactions, and couple it with a transformer-based diffusion model that can synthesize grasps unconditionally or under guidance, using $SE(3)$ Lie-group transformations and DDIM inference. Training is end-to-end with a joint position/rotation noise loss and PyroKi-based fast inverse kinematics, enabling real-time, closed-loop manipulation and cross-embodiment generalization. Across simulation and real-robot experiments, the method achieves state-of-the-art success rates, high inference throughput (up to 41 grasps/s), and substantially lower memory usage than prior work, paving the way for dexterous grasping foundation models with broad embodiment coverage.

Abstract

Dexterous grasping remains a central challenge in robotics due to the complexity of its high-dimensional state and action space. We introduce T(R,O) Grasp, a diffusion-based framework that efficiently generates accurate and diverse grasps across multiple robotic hands. At its core is the T(R,O) Graph, a unified representation that models spatial transformations between robotic hands and objects while encoding their geometric properties. A graph diffusion model, coupled with an efficient inverse kinematics solver, supports both unconditioned and conditioned grasp synthesis. Extensive experiments on a diverse set of dexterous hands show that T(R,O) Grasp achieves average success rate of 94.83%, inference speed of 0.21s, and throughput of 41 grasps per second on an NVIDIA A100 40GB GPU, substantially outperforming existing baselines. In addition, our approach is robust and generalizable across embodiments while significantly reducing memory consumption. More importantly, the high inference speed enables closed-loop dexterous manipulation, underscoring the potential of T(R,O) Grasp to scale into a foundation model for dexterous grasping.

Figures (14)

• Figure 1: Given object point cloud and hand URDF, $\mathcal{T(R,O)}$ Grasp efficiently supports both conditioned and unconditioned grasp synthesis utilizing a graph diffusion model. Compared with $\mathcal{D(R,O)}$ Grasp wei2024dro, our method achieves superior performance with lower memory usage, significantly higher inference speed and throughput.
• Figure 2: Overview of $\mathcal{T(R,O)}$ Grasp: We first define $\mathcal{T(R,O)}$ Graph to represent spatial transformations of robotic links and objects with auxiliary geometry information. Next, we introduce a graph diffusion model that enables both unconditioned and conditioned grasp synthesis.
• Figure 3: Diverse and accurate unconditioned grasp synthesis of $\mathcal{T(R,O)}$ across multiple dexterous hand embodiments.
• Figure 4: Visualization of unconditioned DDIM inference across timesteps. "Pyroki" indicates the grasp after IK with the Pyroki pyroki2025 toolkit.
• Figure 5: grasp synthesis conditioned on initial hand status: Red arrow denotes the palm direction of pose guidance.