Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Analysing the Interplay of Vision and Touch for Dexterous Insertion Tasks

Janis Lenz, Theo Gruner, Daniel Palenicek, Tim Schneider, Jan Peters

TL;DR

This work presents an extensive analysis of the interplay between visual and tactile feedback during dexterous insertion tasks, showing that tactile sensing can greatly enhance success rates on challenging insertions with tight tolerances and varied hole orientations that vision alone cannot solve.

Abstract

Robotic insertion tasks remain challenging due to uncertainties in perception and the need for precise control, particularly in unstructured environments. While humans seamlessly combine vision and touch for such tasks, effectively integrating these modalities in robotic systems is still an open problem. Our work presents an extensive analysis of the interplay between visual and tactile feedback during dexterous insertion tasks, showing that tactile sensing can greatly enhance success rates on challenging insertions with tight tolerances and varied hole orientations that vision alone cannot solve. These findings provide valuable insights for designing more effective multi-modal robotic control systems and highlight the critical role of tactile feedback in contact-rich manipulation tasks.

Analysing the Interplay of Vision and Touch for Dexterous Insertion Tasks

TL;DR

This work presents an extensive analysis of the interplay between visual and tactile feedback during dexterous insertion tasks, showing that tactile sensing can greatly enhance success rates on challenging insertions with tight tolerances and varied hole orientations that vision alone cannot solve.

Abstract

Robotic insertion tasks remain challenging due to uncertainties in perception and the need for precise control, particularly in unstructured environments. While humans seamlessly combine vision and touch for such tasks, effectively integrating these modalities in robotic systems is still an open problem. Our work presents an extensive analysis of the interplay between visual and tactile feedback during dexterous insertion tasks, showing that tactile sensing can greatly enhance success rates on challenging insertions with tight tolerances and varied hole orientations that vision alone cannot solve. These findings provide valuable insights for designing more effective multi-modal robotic control systems and highlight the critical role of tactile feedback in contact-rich manipulation tasks.

Paper Structure

This paper contains 8 sections, 2 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Autonomously Learning Visual-Tactile Peg Insertion in the Real World
  3. Preliminaries.
  4. Hardware setup.
  5. Experimental Results
  6. The impact of tactile feedback during learning.
  7. Explaining the role of touch during an episode.
  8. Conclusion

Figures (3)

  • Figure 1: Dexterous insertion platform using vision and touch. ${\bm{o}}^\text{vis}$ shows the visual observation and ${\bm{o}}^\text{tac}$ the tactile observation.
  • Figure 2: (left) Performances of the visual-only (v) and the visual-tactile (vt) policies trained on 2mm tolerance insertion hole and 0.5mm respectively. (middle) Final performance is reported as the success rate (middle) and the mean rollout length of successful insertions (right) over 20 trials across four varying insertion holes, with increasing evaluation task difficulty from left to right due to decreasing tolerances and increasing hole angles $(\mathrm{tol}, \alpha) \in \{(2, 0), (1, 4), (0.5, 0), (0.5, 4)\}[mm, ~^{\circ}]$.
  • Figure 3: Shapley value analysis over a single exemplary trajectory of the vt 0.5 experiment on the most challenging hole $(\mathrm{tol}=0.5mm, \alpha=4^\circ)$. We report the individual contributions of the four input modalities on the action prediction in x-, y-, and z-direction.