Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Developments and Challenges towards Dexterous and Embodied Robotic Manipulation: A Survey

Gaofeng Li, Ruize Wang, Peisen Xu, Qi Ye, Jiming Chen

TL;DR

The paper surveys the evolution of robotic manipulation from mechanical programming to embodied intelligence, arguing that achieving human-like dexterity requires high-DoF, multi-modal sensing, and end-to-end perception–decision–execution. It consolidates advances in three data-generation paradigms—simulation, human demonstration, and teleoperation—and two learning paradigms—imitation learning and reinforcement learning—highlighting representative datasets, methods, and the remaining gaps. Key contributions include a structured timeline of development, a systematic comparison of data-collection paradigms, and an analysis of IL/RL frameworks with emphasis on their applicability to multi-fingered hands, tactile sensing, and long-horizon tasks. The paper identifies three central challenges—data quality/coverage for high-DoF hands, Sim2Real and Human-to-Robot gaps, and learning frameworks that can generalize and compose skills—and outlines actionable trends toward haptic-augmented teleoperation and hierarchical, multi-modal approaches that bridge perception, action, and reasoning in embodied manipulation.

Abstract

Achieving human-like dexterous robotic manipulation remains a central goal and a pivotal challenge in robotics. The development of Artificial Intelligence (AI) has allowed rapid progress in robotic manipulation. This survey summarizes the evolution of robotic manipulation from mechanical programming to embodied intelligence, alongside the transition from simple grippers to multi-fingered dexterous hands, outlining key characteristics and main challenges. Focusing on the current stage of embodied dexterous manipulation, we highlight recent advances in two critical areas: dexterous manipulation data collection (via simulation, human demonstrations, and teleoperation) and skill-learning frameworks (imitation and reinforcement learning). Then, based on the overview of the existing data collection paradigm and learning framework, three key challenges restricting the development of dexterous robotic manipulation are summarized and discussed.

The Developments and Challenges towards Dexterous and Embodied Robotic Manipulation: A Survey

TL;DR

The paper surveys the evolution of robotic manipulation from mechanical programming to embodied intelligence, arguing that achieving human-like dexterity requires high-DoF, multi-modal sensing, and end-to-end perception–decision–execution. It consolidates advances in three data-generation paradigms—simulation, human demonstration, and teleoperation—and two learning paradigms—imitation learning and reinforcement learning—highlighting representative datasets, methods, and the remaining gaps. Key contributions include a structured timeline of development, a systematic comparison of data-collection paradigms, and an analysis of IL/RL frameworks with emphasis on their applicability to multi-fingered hands, tactile sensing, and long-horizon tasks. The paper identifies three central challenges—data quality/coverage for high-DoF hands, Sim2Real and Human-to-Robot gaps, and learning frameworks that can generalize and compose skills—and outlines actionable trends toward haptic-augmented teleoperation and hierarchical, multi-modal approaches that bridge perception, action, and reasoning in embodied manipulation.

Abstract

Achieving human-like dexterous robotic manipulation remains a central goal and a pivotal challenge in robotics. The development of Artificial Intelligence (AI) has allowed rapid progress in robotic manipulation. This survey summarizes the evolution of robotic manipulation from mechanical programming to embodied intelligence, alongside the transition from simple grippers to multi-fingered dexterous hands, outlining key characteristics and main challenges. Focusing on the current stage of embodied dexterous manipulation, we highlight recent advances in two critical areas: dexterous manipulation data collection (via simulation, human demonstrations, and teleoperation) and skill-learning frameworks (imitation and reinforcement learning). Then, based on the overview of the existing data collection paradigm and learning framework, three key challenges restricting the development of dexterous robotic manipulation are summarized and discussed.

Paper Structure

This paper contains 12 sections, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. The Historical Stages of Robotic Manipulation
  3. The Challenges in Dexterous Manipulation: from Gripper to Muti-Fingered Hand
  4. The Data Collection Paradigm for Dexterous Manipulation
  5. Data Generation Based on Simulation Platform
  6. Data Collection from Human Demonstration
  7. Data Collection from Teleoperation Demonstration
  8. The Learning Framework for Dexterous Manipulation Skills
  9. Imitation Learning
  10. Reinforcement Learning
  11. Open Challenges and New Trends
  12. Conclusion

Figures (5)

  • Figure 1: The hardware progress of end-effector and manipulator for robotic manipulation.
  • Figure 2: The development process of robotic manipulation. There are three stages, including mechanical programming, closed-loop control, and embodied intelligence.
  • Figure 3: The challenges in interacting with physical world. The robot has to face more and more complex manipulated objects and diverse manipulation types.
  • Figure 4: Illustration of the human-to-robot gap. The comparison of (a) human hand, (b) Allegro Hand and (c) Shadow Hand. It is obvious that the Allegro Hand's fingers are mach larger than human and the Shadow Hand has a huge driven box, which bring negative effects to the Arm-Hand system.
  • Figure 5: Challenges and new trends in robotic dexterous manipulation learning