Towards Feasible Dynamic Grasping: Leveraging Gaussian Process Distance Field, SE(3) Equivariance and Riemannian Mixture Models

TL;DR

A novel approach to improve robotic grasping in dynamic environments by integrating Gaussian Process Distance Fields, SE(3) equivariant networks, and Riemannian Mixture Models is introduced, presenting a promising solution for enhancing robotic grasping capabilities in real-world scenarios.

Abstract

This paper introduces a novel approach to improve robotic grasping in dynamic environments by integrating Gaussian Process Distance Fields (GPDF), SE(3) equivariant networks, and Riemannian Mixture Models. The aim is to enable robots to grasp moving objects effectively. Our approach comprises three main components: object shape reconstruction, grasp sampling, and implicit grasp pose selection. GPDF accurately models the shape of objects, which is essential for precise grasp planning. SE(3) equivariance ensures that the sampled grasp poses are equivariant to the object's pose changes, enhancing robustness in dynamic scenarios. Riemannian Gaussian Mixture Models are employed to assess reachability, providing a feasible and adaptable grasping strategies. Feasible grasp poses are targeted by novel task or joint space reactive controllers formulated using Gaussian Mixture Models and Gaussian Processes. This method resolves the challenge of discrete grasp pose selection, enabling smoother grasping execution. Experimental validation confirms the effectiveness of our approach in generating feasible grasp poses and achieving successful grasps in dynamic environments. By integrating these advanced techniques, we present a promising solution for enhancing robotic grasping capabilities in real-world scenarios.

Figures (7)

• Figure 1: Experimental setup used to evaluate our feasible dynamic grasping framework (left: static object, right: dynamic object). An Intel RealSense camera mounted on the robot's end-effector is used to obtain a partial point cloud of the object and its pose (optitrack is used to track the object in dynamic scenarios) with no prior knowledge about the object category.
• Figure 2: Feasible Dynamic Grasping Framework pipeline. Given a partial point cloud of an object and its pose, we reconstruct the shape using the Gaussian Process Distance Field (Section \ref{['sec:gpis']}) with a refined distance metric as in Section \ref{['sec:distance_field']}. Next, we sample poses from an SE(3) equivariant grasp sampler (Section \ref{['sec:grasp_sampler']}). Finally, the discrete set of feasible grasp poses is used to i) construct a continuous Riemannian GMM for dynamic task-space control (Section \ref{['sec:task_control']}) or ii) convert to joint-space via IK to construct a continuous GP for dynamic control in joint-space (Section \ref{['sec:joint_control']}). The grasp sampling and controller components can operate asynchronously, allowing for real-time adaptation.
• Figure 3: A 1D slice of the distance field of a sphere point cloud. The image on the left shows a comparison between the real SDF and the initial estimation from the GPDF. On the right, the image shows an improved estimation after five iterations of gradient descent, employing the ray marching concept.
• Figure 4: The reconstruction is done using the Gaussian process. The initial two images on the left display the mesh of a mustard bottle. In the subsequent two images, partial point clouds of the object are displayed. Finally, the last image shows the reconstructed point cloud of the object. Areas with low shape uncertainty are colored blue, while regions with high shape uncertainty are highlighted in red.
• Figure 5: Architecture of the data-driven grasp sampler, numbers (*) denote the respective number of layers in each component.