DORA: Object Affordance-Guided Reinforcement Learning for Dexterous Robotic Manipulation

TL;DR

This work tackles efficient dexterous manipulation by incorporating object affordances into both grasp generation and reinforcement learning. It introduces an object affordance-guided pipeline that produces functionally grounded grasp candidates and uses an affordance-aware reward to shape policy learning, evaluated on cube, jug, and hammer tasks. The results show notable improvements in task success and sample efficiency, with average gains around 15.4% and faster convergence, while also highlighting algorithm-dependent effects (PPO vs SAC). This approach advances semantically grounded, generalizable manipulation policies and points to future refinements in realistic contact modeling and mesh-aware training.

Abstract

Dexterous robotic manipulation remains a longstanding challenge in robotics due to the high dimensionality of control spaces and the semantic complexity of object interaction. In this paper, we propose an object affordance-guided reinforcement learning framework that enables a multi-fingered robotic hand to learn human-like manipulation strategies more efficiently. By leveraging object affordance maps, our approach generates semantically meaningful grasp pose candidates that serve as both policy constraints and priors during training. We introduce a voting-based grasp classification mechanism to ensure functional alignment between grasp configurations and object affordance regions. Furthermore, we incorporate these constraints into a generalizable RL pipeline and design a reward function that unifies affordance-awareness with task-specific objectives. Experimental results across three manipulation tasks - cube grasping, jug grasping and lifting, and hammer use - demonstrate that our affordance-guided approach improves task success rates by an average of 15.4% compared to baselines. These findings highlight the critical role of object affordance priors in enhancing sample efficiency and learning generalizable, semantically grounded manipulation policies. For more details, please visit our project website https://sites.google.com/view/dora-manip.

Figures (5)

• Figure 1: Overview of the proposed object affordance-guided reinforcement learning framework. Object affordance maps are used to generate semantically meaningful functional grasp candidates. These candidates serve as constraints and priors to guide the RL policy, which is further optimized using an object affordance-aware reward. The resulting policy enables dexterous manipulation that is functionally grounded and task-relevant.
• Figure 2: Overview of the object affordance-guided reinforcement learning pipeline for various dexterous manipulation tasks. Functional grasp candidates are first generated based on object affordance information. Feasible candidates are then filtered through motion planning checks and used as constraints during RL training. Additionally, an affordance-aware reward is designed to guide policy learning across tasks such as pick-and-place, functional grasping, and in-hand reorientation.
• Figure 3: Examples of object affordances used in the task.
• Figure 4: Performance comparison across three tasks based on optimal policy. Task 1: Cube grasping and lifting, Task 2: Jug functional grasping and lifting, Task 3: Hammer Grasping and Re-Orientation.
• Figure 5: Scene of task 4 for opening microwave and picking, placing cube.