A Multi-Layer Sim-to-Real Framework for Gaze-Driven Assistive Neck Exoskeletons

TL;DR

This work uses virtual reality to collect coupled eye and head movement data from healthy individuals to train models capable of predicting head movement based solely on eye gaze, and proposes a novel multi-layer controller selection framework that effectively rejects poor-performing controllers early.

Abstract

Dropped head syndrome, caused by neck muscle weakness from neurological diseases, severely impairs an individual's ability to support and move their head, causing pain and making everyday tasks challenging. Our long-term goal is to develop an assistive powered neck exoskeleton that restores natural movement. However, predicting a user's intended head movement remains a key challenge. We leverage virtual reality (VR) to collect coupled eye and head movement data from healthy individuals to train models capable of predicting head movement based solely on eye gaze. We also propose a novel multi-layer controller selection framework, where head control strategies are evaluated across decreasing levels of abstraction -- from simulation and VR to a physical neck exoskeleton. This pipeline effectively rejects poor-performing controllers early, identifying two novel gaze-driven models that achieve strong performance when deployed on the physical exoskeleton. Our results reveal that no single controller is universally preferred, highlighting the necessity for personalization in gaze-driven assistive control. Our work demonstrates the utility of VR-based evaluation for accelerating the development of intuitive, safe, and personalized assistive robots.

Figures (10)

• Figure 1: We propose to leverage multiple levels of abstraction and emulation to assist in efficiently finding useful gaze-driven neck exoskeleton controllers. As the initial controller pool moves down through each layer, poor controllers are rejected from further study. Our goal is to arrive at a final set of controller designs suitable for restoring head-neck motions.
• Figure 2: Two examples of the Vector Parameterized control law for either pitch or yaw directions. The $v$ and $c$ values are equal for both examples. The control law transitions from a deadzone with zero velocity to a line with proportional velocity. Gray dotted lines mark inflection point locations of the plots. Blue dashed lines show what the control law transitions to and from. The Purple curve on the right has a larger but stricter deadzone.
• Figure 3: Trajectories of an autoregressive rollout for one MLP and two LSTM variations. The y-axis shows the $x$ (blue), $y$ (orange), and $z$ (green) values of the normalized direction of the head where $(0, 0, 1)$ represents the forward direction. The x-axis shows the time step $t$ throughout the trajectory. Solid lines represent the direction of the simulated, rotated "virtual head" during the experiment. Dashed lines represent the ground-truth direction of the head as seen in the dataset. (Top) An MLP with two intermediate hidden layers each of size 16. (Middle) An LSTM with a context vector of size 2. (Bottom) An LSTM with a context vector of size 16.
• Figure 4: Distribution of mean squared error (lower is better) between imputed head direction and ground-truth head direction for various models over trajectories from the held-out test set. Results are grouped by task.
• Figure 5: Example target trajectories. (a) The target in the Linear Smooth Pursuit task moves in straight lines of random direction and distance. (b) The target in the Arc Smooth Pursuit task moves in circular lines of random direction, distance, and curvature. Both pursuit tasks feature only a single target. (c) Each of the three targets in the Rapid Visual Search task starts some distance from the participant and moves towards the participant. (d) In the Rapid Visual Search Avoidance task there are three targets to fixate on (blue) and three blocking targets to not fixate on (yellow). Each target and blocking cube starts from a plane some distance from the participant and moves towards the participant. The dashed line represents a horizon.