AffordanceGrasp-R1:Leveraging Reasoning-Based Affordance Segmentation with Reinforcement Learning for Robotic Grasping

TL;DR

AffordanceGrasp-R1 advances robotic grasping by marrying reasoning-based affordance segmentation with reinforcement learning. It introduces a three-stage post-training pipeline—CoT-based cold-start SFT, GRPO RL refinement, and LoRA-tuned SAM 2—for high-fidelity, instruction-conditioned affordance masks while preserving a global scene point cloud for grasp candidate generation. By applying instruction-conditioned masks to filter 6-DoF grasp proposals, the method maintains global geometry awareness and robust grounding under language constraints. On the RAGNet benchmark and in strict zero-shot real-world experiments, AffordanceGrasp-R1 achieves state-of-the-art performance and strong generalization across complex manipulation tasks.

Abstract

We introduce AffordanceGrasp-R1, a reasoning-driven affordance segmentation framework for robotic grasping that combines a chain-of-thought (CoT) cold-start strategy with reinforcement learning to enhance deduction and spatial grounding. In addition, we redesign the grasping pipeline to be more context-aware by generating grasp candidates from the global scene point cloud and subsequently filtering them using instruction-conditioned affordance masks. Extensive experiments demonstrate that AffordanceGrasp-R1 consistently outperforms state-of-the-art (SOTA) methods on benchmark datasets, and real-world robotic grasping evaluations further validate its robustness and generalization under complex language-conditioned manipulation scenarios.

Figures (8)

• Figure 1: Performance comparison between AffordanceNet wu2025ragnet and our method on grasping datasets, including GraspNet fang2020graspnet, HANDAL guo2023handal, and 3DOI qian2023understanding.
• Figure 2: Visualization of the instruction, bounding box, point, and mask annotations.
• Figure 3: 3D IOU matching for grasp selection: We compute the 3D IoU between the gripper closing volume and the semantics-aligned target region induced by the predicted affordance subcloud $\hat{S}$. The red region indicates the overlap volume used to compute the 3D IoU score.
• Figure 4: Qualitative comparison of affordance segmentation results on the main subset wu2025ragnet.
• Figure 5: Qualitative comparison of affordance segmentation on the reasoning-based subset. AffordanceGrasp-R1 consistently outperforms AffordanceNet in segmentation accuracy.