Embodied Supervision: Haptic Display of Automation Command to Improve Supervisory Performance

TL;DR

This work investigates whether providing a supervisor with a copy of the operator's control command $u(t)$ improves supervisory performance in a shared-control setting. Through a human-in-the-loop experiment comparing no-copy, visual-copy, and haptic-copy conditions, the study finds that haptic access to $u(t)$ yields higher target identification accuracy and shorter decision delays than both no-copy and visual-copy, with strong correspondence to operator performance. The results support embodied supervision, suggesting that kinesthetic information aligned with the supervisor's own motor experience can enhance situation awareness and decision speed, with implications for safer and more efficient automated systems. The authors discuss training effects, potential confounds (e.g., mechanical resistance), and outline future work extending the approach to autonomous agents and multi-agent supervisory contexts.

Abstract

A human operator using a manual control interface has ready access to their own command signal, both by efference copy and proprioception. In contrast, a human supervisor typically relies on visual information alone. We propose supplying a supervisor with a copy of the operators command signal, hypothesizing improved performance, especially when that copy is provided through haptic display. We experimentally compared haptic with visual access to the command signal, quantifying the performance of N equals 10 participants attempting to determine which of three reference signals was being tracked by an operator. Results indicate an improved accuracy in identifying the tracked target when haptic display was available relative to visual display alone. We conjecture the benefit follows from the relationship of haptics to the supervisor's own experience, perhaps muscle memory, as an operator.

Figures (5)

• Figure 1: Block Diagram: (A): The supervisor without access to $u(t)$ must use $y(t)$ as the best estimate of $r(t)$, whereas (B): a supervisor able to observe $u(t)$ can construct a superior estimate $\hat{r}(t)$ by combining a scaled copy of $u(t)$ with $y(t)$.
• Figure 2: Simulation Results: When the supervisor does not have access to $u(t)$, their best estimate of the reference signal $r(t)$ (blue) is $y(t)$ itself (yellow). On the other hand, when the supervisor has access to $u(t)$, their best estimate of $r(t)$ is $\hat{r}(t)$ (red). As is evident in the plots, $\hat{r}(t)$ has a much smaller phase difference with $r(t)$ than $y(t)$ does, suggesting that $\hat{r}(t)$ is more effective at accounting for the controller's delay and phase lag in the integrator plant.
• Figure 3: Experimental Setup: The Operator Station is on the left and Supervisor/Participant Station is on the right.
• Figure 4: Operator Performance: The target visible to the operator followed reference signal $r_1$ until $t=4.04s$, then $r_2$ until $t=8.84s$, then $r_3$ until $t=10.50s$, then $r_1$ until $t=12.68s$, and thereafter $r_3$. The operator's performance in tracking the selected reference signal can be seen in the dashed black line $y(t)$. The supervisor's keypresses are indicated in the overlay staircase plot.
• Figure 5: Target Selection Accuracy and Target Selection Delay by condition in Box and Whiskers Plots. Significant differences were found in Target Selection Accuracy between the uVisual and uHaptic conditions.