Contact Coverage-Guided Exploration for General-Purpose Dexterous Manipulation

TL;DR

Experimental results show that CCGE substantially improves training efficiency and success rates over existing exploration methods, and that the contact patterns learned with CCGE transfer robustly to real-world robotic systems.

Abstract

Deep Reinforcement learning (DRL) has achieved remarkable success in domains with well-defined reward structures, such as Atari games and locomotion. In contrast, dexterous manipulation lacks general-purpose reward formulations and typically depends on task-specific, handcrafted priors to guide hand-object interactions. We propose Contact Coverage-Guided Exploration (CCGE), a general exploration method designed for general-purpose dexterous manipulation tasks. CCGE represents contact state as the intersection between object surface points and predefined hand keypoints, encouraging dexterous hands to discover diverse and novel contact patterns, namely which fingers contact which object regions. It maintains a contact counter conditioned on discretized object states obtained via learned hash codes, capturing how frequently each finger interacts with different object regions. This counter is leveraged in two complementary ways: (1) to assign a count-based contact coverage reward that promotes exploration of novel contact patterns, and (2) an energy-based reaching reward that guides the agent toward under-explored contact regions. We evaluate CCGE on a diverse set of dexterous manipulation tasks, including cluttered object singulation, constrained object retrieval, in-hand reorientation, and bimanual manipulation. Experimental results show that CCGE substantially improves training efficiency and success rates over existing exploration methods, and that the contact patterns learned with CCGE transfer robustly to real-world robotic systems. Project page is https://contact-coverage-guided-exploration.github.io.

Figures (12)

• Figure 1: CCGE is a general exploration method, which utilizes contact coverage over the target object to guide dexterous hands towards under-explored object regions. CCGE achieves strong performance with training efficiency in diverse manipulation tasks.
• Figure 2: Overview of CCGE. CCGE proposes a contact coverage-guided exploration method that explicitly models hand–object interactions, consisting of three key components: a learned state hashing module that discretizes continuous object states into compact state clusters, a contact coverage counter that records state-conditioned finger–region interactions, and a structured exploration reward. The exploration reward is further decomposed into a contact coverage reward, which encourages exploration of under-explored contact regions after contact occurs, and a pre-contact energy-based reaching reward, which guides the policy toward unexplored object regions, facilitating efficient contact discovery before physical interaction occurs. The current object state and the goal state are visualized as colored point clouds, with colors indicating different object surface regions.
• Figure 3: Hand Keypoint Representation. We represent the dexterous hand fingers using a sparse set of keypoints (visualized as red spheres).
• Figure 4: Learning Curves of 4 Dexterous Manipulation Tasks. Our method, CCGE, leverages contact-guided exploration to achieve higher sample efficiency and success rates, particularly in "hard exploration" tasks like Constrained Object Retrieval where baselines fail.
• Figure 5: Visualization of default initialization and pre-contact initialization in Constrained Object Retrieval.