Human Impedance Modulation to Improve Visuo-Haptic Perception

Abstract

Humans activate muscles to shape the mechanical interaction with their environment, but can they harness this control mechanism to best sense the environment? We investigated how participants adapt their muscle activation to visual and haptic information when tracking a randomly moving target with a robotic interface. The results exhibit a differentiated effect of these sensory modalities, where participants' muscle cocontraction increases with the haptic noise and decreases with the visual noise, in apparent contradiction to previous results. These results can be explained, and reconciled with previous findings, when considering muscle spring like mechanics, where stiffness increases with cocontraction to regulate motion guidance. Increasing cocontraction to more closely follow the motion plan favors accurate visual over haptic information, while decreasing it avoids injecting visual noise and relies on accurate haptic information. We formulated this active sensing mechanism as the optimization of visuo-haptic information and effort. This OIE model can explain the adaptation of muscle activity to unimodal and multimodal sensory information when interacting with fixed or dynamic environments, or with another human, and can be used to optimize human-robot interaction.

Figures (4)

• Figure 1: Experiment setup and protocol. A: Participants were asked to track a randomly moving target with noisy visual feedback and in some conditions were connected to the human-like tracking controller of Takagi2017. B,C: Experiment protocol of separate visual or haptic feedback experiment, with each block consisting of nine trials. The 13 participants received only visual/haptic feedback in random order and each with random noise level (B). Another 22 participants experienced nine integrated visual and haptic conditions presented in a random order (C). (D) illustrates the mechanical modelling scheme of the human-robot interaction with visual and haptic noise.
• Figure 2: Results of solely visual/haptic feedback experiment. A&B: Evolution of tracking error and cocontraction with visual and/or haptic feedback, where error bars represent standard error. C&D: The mean and standard error of tracking error and cocontraction for all subjects during the last four trials. In the visual feedback condition (A&C), with increasing visual noise the tracking error increases while the cocontraction decreases. In the haptic feedback condition (B&D), there is a decaying trend of both tracking error and cocontraction over the trials and no clear difference among noise conditions.
• Figure 3: Results of tracking with combined visual and haptic noise. A: Evolution of tracking error and cocontraction with visual and/or haptic feedback, with error bars representing the standard error. The tracking error saturates in the initial solo trials, and increases with both visual and haptic noise during interactive trials. Cocontraction shows a slower decrease across trials. B: Mean and standard error of tracking error and cocontraction for all subjects during the last four trials. Tracking error increases with either visual or haptic noise, while muscle cocontraction increases with haptic noise and decreases with visual noise. C: Muscle cocontraction and reciprocal activation waveforms along with their frequency spectrum. Muscle cocontraction remains relatively constant, while reciprocal activation changes synchronised with the movement, as is confirmed by their spectra.
• Figure 4: Simulation results of normalized cocontraction. (A) Comparison between the tracking error minimization (TEM) model and our optimal information and effort (OIE) model across the nine experiment conditions. The OIE model predicted normalized cocontraction values closely aligned with the experimental data, while the TEM model produced large prediction errors. (B) OIE model predictions of normalized cocontraction as a function of visual and haptic noise levels. Black dots represent the recorded average cocontraction of 20 participants in the final trial. Red dots represent the fitted data from the OIE model, and green dots the predicted data. The blue dots show the model's predicted cocontraction in unobserved noise conditions.