DexEvolve: Evolutionary Optimization for Robust and Diverse Dexterous Grasp Synthesis

TL;DR

This work proposes a scalable generate-and-refine pipeline for synthesizing large-scale, diverse, and physically feasible grasps, and distill the refined grasp distribution into a diffusion model for robust real-world deployment, and highlights the role of diversity for both effective training and during deployment.

Abstract

Dexterous grasping is fundamental to robotics, yet data-driven grasp prediction heavily relies on large, diverse datasets that are costly to generate and typically limited to a narrow set of gripper morphologies. Analytical grasp synthesis can be used to scale data collection, but necessary simplifying assumptions often yield physically infeasible grasps that need to be filtered in high-fidelity simulators, significantly reducing the total number of grasps and their diversity. We propose a scalable generate-and-refine pipeline for synthesizing large-scale, diverse, and physically feasible grasps. Instead of using high-fidelity simulators solely for verification and filtering, we leverage them as an optimization stage that continuously improves grasp quality without discarding precomputed candidates. More specifically, we initialize an evolutionary search with a seed set of analytically generated, potentially suboptimal grasps. We then refine these proposals directly in a high-fidelity simulator (Isaac Sim) using an asynchronous, gradient-free evolutionary algorithm, improving stability while maintaining diversity. In addition, this refinement stage can be guided toward human preferences and/or domain-specific quality metrics without requiring a differentiable objective. We further distill the refined grasp distribution into a diffusion model for robust real-world deployment, and highlight the role of diversity for both effective training and during deployment. Experiments on a newly introduced Handles dataset and a DexGraspNet subset demonstrate that our approach achieves over 120 distinct stable grasps per object (a 1.7-6x improvement over unrefined analytical methods) while outperforming diffusion-based alternatives by 46-60\% in unique grasp coverage.

Figures (20)

• Figure 1: Visualization of Generated Grasps using our Evolutionary Refinement visualized for two Handles and one object asset. Our evolutionary approach generates diverse, physically stable grasps by refining candidates directly within high-fidelity simulation.
• Figure 2: Method Overview. Given object meshes, we first generate an initial set of analytical grasp proposals. We then refine these candidates in a high-fidelity simulator using an asynchronous gradient-free evolutionary loop (selection, crossover, mutation, and physics-based evaluation) to obtain a diverse set of physically feasible grasps. Finally, we distill the refined grasp distribution into a diffusion model conditioned on noisy pointcloud observations, enabling efficient sampling at inference time under partial observability.
• Figure 3: Contact Point Selection for Dexterous Grasping. Two views of a dexterous hand (Xhand) grasping a cup from the Objects dataset. Light blue points indicate candidate contact points on the object surface that lie within the contact distance threshold, while dark blue points denote points outside this threshold. Green points represent the final active contact points selected via farthest point sampling (FPS), which are used to compute grasping commands through the contact Jacobian.
• Figure 4: Selection of our IKEA Handles Dataset. We provide 90 geometrically distinct handle assets with multiple texture variations, totaling 190 unique simulation-ready assets. Each handle includes high-resolution collision geometry, realistic textures, and articulated joints with compliant degrees of freedom to support stable grasp synthesis. All assets are based on commercially available products from ikea.com and can be directly used in Isaac Sim.
• Figure 5: Grasp Quality and Diversity Comparison. Radar plots comparing synthesized grasp distributions across different generation strategies. We report grasp stability (success rate), uniqueness of successful grasps at different thresholds (DSG@2cm/DSG@20cm), and diversity via marginal entropies of hand position, orientation, and joint angles. Overall, evolutionary refinement improves stability while maintaining high diversity across both datasets.