Evaluation of Human-Robot Interfaces based on 2D/3D Visual and Haptic Feedback for Aerial Manipulation

TL;DR

Results show that both 3D MR vision and haptic feedback improve the operator’s dexterity in the considered teleoperated aerial interaction tasks, Nevertheless, pilot experience remains the most significant factor.

Abstract

Most telemanipulation systems for aerial robots provide the operator with only 2D screen visual information. The lack of richer information about the robot's status and environment can limit human awareness and, in turn, task performance. While the pilot's experience can often compensate for this reduced flow of information, providing richer feedback is expected to reduce the cognitive workload and offer a more intuitive experience overall. This work aims to understand the significance of providing additional pieces of information during aerial telemanipulation, namely (i) 3D immersive visual feedback about the robot's surroundings through mixed reality (MR) and (ii) 3D haptic feedback about the robot interaction with the environment. To do so, we developed a human-robot interface able to provide this information. First, we demonstrate its potential in a real-world manipulation task requiring sub-centimeter-level accuracy. Then, we evaluate the individual effect of MR vision and haptic feedback on both dexterity and workload through a human subjects study involving a virtual block transportation task. Results show that both 3D MR vision and haptic feedback improve the operator's dexterity in the considered teleoperated aerial interaction tasks. Nevertheless, pilot experience remains the most significant factor.

Figures (11)

• Figure 1: On the top: Local teleoperation environment with haptic joystick and hologram of the robot in its environment. On the bottom: Remote environment with the real robot.
• Figure 2: Detailed interactions between the different components of the teleoperation framework.
• Figure 3: Experimental setup for force exertion \ref{['fig:experiment-a']} and insertion task \ref{['fig:experiment-b']}, with corresponding virtual MR holograms \ref{['fig:experiment-c']} and \ref{['fig:experiment-d']}.
• Figure 4: Experiment 1 - Measured position in $\mathcal{F}_{W}$ and estimated forces in $\mathcal{F}_{W}$ while pushing the wheeled object. From left to right the highlighted moments are: before the interaction, the first physical contact, a slight discontinuity of contact, and the final contact disengagement.
• Figure 5: Experiment 2 - Measured position in the world frame $\mathcal{F}_{W}$ and estimated forces in the body frame $\mathcal{F}_{B}$ during the peg-in-hole task. From left to right the highlighted moments are: before the interaction, the shocks before the insertion, the reach of the hole's end, and the exit from the hole.