Kinesthetic Teaching in Robotics: a Mixed Reality Approach

TL;DR

A novel communicative interface that uses Mixed Reality as a medium to perform Kinesthetic Teaching on any robotic platform is proposed, and this approach is evaluated in a user study involving multiple subjects and two different robots.

Abstract

As collaborative robots become more common in manufacturing scenarios and adopted in hybrid human-robot teams, we should develop new interaction and communication strategies to ensure smooth collaboration between agents. In this paper, we propose a novel communicative interface that uses Mixed Reality as a medium to perform Kinesthetic Teaching (KT) on any robotic platform. We evaluate our proposed approach in a user study involving multiple subjects and two different robots, comparing traditional physical KT with holographic-based KT through user experience questionnaires and task-related metrics.

Figures (6)

• Figure 1: An experimenter in the middle of a holographic KT session with the Tiago++ robot. By interacting with and manipulating the grey holographic sphere, superimposed on the digital robot's wrist and here highlighted via a red circle, the user can teach actions to the robot teammate using gestures and voice.
• Figure 2: Overview of the proposed architecture implementing holographic KT, extending the framework detailed in macciò2023ricomr.
• Figure 3: An experimenter interacting with Baxter during physical KT session. The operator drives the robot's arm through gestural interaction, teaching the sequence of pick-and-place actions needed to complete the stacking task.
• Figure 4: Histograms depicting the number of cubes successfully stacked by the robots during the playback phase, in the two experimental conditions.
• Figure 5: Differential distributions depicting the temporal overhead introduced by the MR medium when performing KT under C2.