Grasp the Graph (GtG) 2.0: Ensemble of Graph Neural Networks for High-Precision Grasp Pose Detection in Clutter

TL;DR

Grasp the Graph 2.0 (GtG 2.0) tackles grasp pose detection in clutter by combining a fast, geometry-driven candidate generator with an ensemble of lightweight Graph Neural Networks. By representing each grasp candidate as a graph that includes both inside and outside points, GtG 2.0 achieves robust, high-precision scoring while maintaining a small parameter footprint ($ ext{ensemble size} imes 0.11$M per model). The authors reformulated GraspNet-1Billion labels with a friction-based quality metric and demonstrate up to $35\%$ AP improvement over baselines, ranking among the top methods while using far fewer parameters. Real-robot experiments with a $4$-DoF setup reach $91\%$ grasp success and $100\%$ scene completion, validating practicality in cluttered, real-world settings. Limitations include performance on novel scenes and the need for more advanced candidate generation, motivating future work toward end-to-end single-stage frameworks and enhanced generalization.

Abstract

Grasp pose detection in cluttered, real-world environments remains a significant challenge due to noisy and incomplete sensory data combined with complex object geometries. This paper introduces Grasp the Graph 2.0 (GtG 2.0) method, a lightweight yet highly effective hypothesis-and-test robotics grasping framework which leverages an ensemble of Graph Neural Networks for efficient geometric reasoning from point cloud data. Building on the success of GtG 1.0, which demonstrated the potential of Graph Neural Networks for grasp detection but was limited by assumptions of complete, noise-free point clouds and 4-Dof grasping, GtG 2.0 employs a conventional Grasp Pose Generator to efficiently produce 7-Dof grasp candidates. Candidates are assessed with an ensemble Graph Neural Network model which includes points within the gripper jaws (inside points) and surrounding contextual points (outside points). This improved representation boosts grasp detection performance over previous methods using the same generator. GtG 2.0 shows up to a 35% improvement in Average Precision on the GraspNet-1Billion benchmark compared to hypothesis-and-test and Graph Neural Network-based methods, ranking it among the top three frameworks. Experiments with a 3-Dof Delta Parallel robot and Kinect-v1 camera show a success rate of 91% and a clutter completion rate of 100%, demonstrating its flexibility and reliability.

Figures (6)

• Figure 1: Overview of the GtG 2.0 pipeline. (1) Grasp candidates are generated using the GPG algorithm. (2) Each candidate’s local region is segmented into inside points and outside points, which are sampled and converted into a graph using $k$-NN. (3) The resulting graph is passed through an ensemble of GNNs. Each model predicts a grasp score, and the final score is the average of all ensemble outputs.
• Figure 2: Architecture of the GNN-based grasp scoring network. Each input graph, composed of inside points and outside points, is passed through a position and label encoder, a stack of SAGEConv layers, and an element-wise transformation block. Node-level features are aggregated via global max pooling and fed into a final predictor MLP to generate a score. An ensemble of five such GNNs processes the same graph independently, and their outputs are averaged to produce the final grasp score.
• Figure 3: Illustration of label discrepancies in GraspNet-1Billion: the horizontal axis shows original GraspNet-1Billion scores, while the vertical axis represents recalculated quality. Many grasps labeled "valid" by GraspNet (upper right region) are deemed collision-prone or infeasible upon reevaluation.
• Figure 4: Histogram of the curated dataset's grasp score distribution, separated by sensor type (RealSense and Kinect). The y-axis is on a logarithmic scale. Scores on the x-axis categorize the grasps: -1.0 indicates a definite collision, -0.5 marks a low-quality or infeasible grasp, and scores from 0.0 to 1.0 represent valid grasps of increasing quality.
• Figure 5: Visualization of the final 50 grasps detected by GtG 2.0 and evaluated by the GraspNet benchmark. Each subimage displays 25 top-scoring grasps overlaid on the scene from different viewpoints. Warmer colors denote higher predicted grasp scores, while cooler colors correspond to lower scores. Splitting the grasps into two groups of 25 improves clarity by preserving object detail compared to plotting all 50 simultaneously.