Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

All the Feels: A dexterous hand with large-area tactile sensing

Raunaq Bhirangi, Abigail DeFranco, Jacob Adkins, Carmel Majidi, Abhinav Gupta, Tess Hellebrekers, Vikash Kumar

TL;DR

This work tackles the scarcity of affordable, robust dexterous robotic hands with full-surface tactile sensing by introducing the D'Manus, a low-cost 10-DoF hand equipped with ReSkin large-area tactile sensing. The authors demonstrate the platform's effectiveness for real-world robot learning through dexterity tests, material and texture softness perception, and a tactile-based bin-sorting task, supported by a MuJoCo-based simulation and a data-driven ReSkin model. Key contributions include hardware design and open-source release, extensive long-duration testing, and robust tactile perception models that generalize to unseen objects. The results underscore the potential of large-area tactile sensing to enable scalable, tissue-like manipulation and pave the way for multimodal sensor fusion in tactile-rich robotic tasks.

Abstract

High cost and lack of reliability has precluded the widespread adoption of dexterous hands in robotics. Furthermore, the lack of a viable tactile sensor capable of sensing over the entire area of the hand impedes the rich, low-level feedback that would improve learning of dexterous manipulation skills. This paper introduces an inexpensive, modular, robust, and scalable platform -- the DManus -- aimed at resolving these challenges while satisfying the large-scale data collection capabilities demanded by deep robot learning paradigms. Studies on human manipulation point to the criticality of low-level tactile feedback in performing everyday dexterous tasks. The DManus comes with ReSkin sensing on the entire surface of the palm as well as the fingertips. We demonstrate effectiveness of the fully integrated system in a tactile aware task -- bin picking and sorting. Code, documentation, design files, detailed assembly instructions, trained models, task videos, and all supplementary materials required to recreate the setup can be found on https://sites.google.com/view/roboticsbenchmarks/platforms/dmanus.

All the Feels: A dexterous hand with large-area tactile sensing

TL;DR

This work tackles the scarcity of affordable, robust dexterous robotic hands with full-surface tactile sensing by introducing the D'Manus, a low-cost 10-DoF hand equipped with ReSkin large-area tactile sensing. The authors demonstrate the platform's effectiveness for real-world robot learning through dexterity tests, material and texture softness perception, and a tactile-based bin-sorting task, supported by a MuJoCo-based simulation and a data-driven ReSkin model. Key contributions include hardware design and open-source release, extensive long-duration testing, and robust tactile perception models that generalize to unseen objects. The results underscore the potential of large-area tactile sensing to enable scalable, tissue-like manipulation and pave the way for multimodal sensor fusion in tactile-rich robotic tasks.

Abstract

High cost and lack of reliability has precluded the widespread adoption of dexterous hands in robotics. Furthermore, the lack of a viable tactile sensor capable of sensing over the entire area of the hand impedes the rich, low-level feedback that would improve learning of dexterous manipulation skills. This paper introduces an inexpensive, modular, robust, and scalable platform -- the DManus -- aimed at resolving these challenges while satisfying the large-scale data collection capabilities demanded by deep robot learning paradigms. Studies on human manipulation point to the criticality of low-level tactile feedback in performing everyday dexterous tasks. The DManus comes with ReSkin sensing on the entire surface of the palm as well as the fingertips. We demonstrate effectiveness of the fully integrated system in a tactile aware task -- bin picking and sorting. Code, documentation, design files, detailed assembly instructions, trained models, task videos, and all supplementary materials required to recreate the setup can be found on https://sites.google.com/view/roboticsbenchmarks/platforms/dmanus.
Paper Structure (29 sections, 11 figures, 8 tables)

This paper contains 29 sections, 11 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Dexterous Hands and data-driven learning
  4. Tactile sensing
  5. Platform and System Details
  6. The Hand: Construction and Interfacing
  7. Large-area Exteroceptive Sensing: ReSkin
  8. Control and Proprioceptive Sensing
  9. Software
  10. Experiments
  11. Tactile Perception: Data Collection and Modeling
  12. Data Collection
  13. Model Learning
  14. Results
  15. Dexterity of the D'Manus
  16. ...and 14 more sections

Figures (11)

  • Figure 1: The D'Manus -- a low-cost, 10 DoF, reliable prehensile hand with all-over ReSkin bhirangi2021reskin sensing.
  • Figure 2: Anatomy of the D'Manus hand: The D'Manus is actuated at joint level using Dynamixel XM430-210 smart actuators. ReSkin sensors are integrated with the fingertips and the palm. Each fingertip sensor is comprised of 8 magnetometers while the palm sensor consists of 32 magnetometers for a total of 56 magnetometers. Sensor and motor interfacing components are housed in the core of the hand.
  • Figure 3: Simulated D'Manus
  • Figure 4: Data collection setup: Tactile data is collected by placing the object on the palm and executing a human-scripted interaction policy for motor babble.
  • Figure 5: Sample ReSkin data: Visualization of tactile data from two of the fingers while interacting with the loofah in Fig. \ref{['fig:data-collection-setup']}.
  • ...and 6 more figures